Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.949950984s FAIL (2025-09-23 16:31:06 UTC) orbiter warning: context deadline exceeded Duration: 5.949980239s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.19186591s FAIL (2025-09-23 16:30:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.191894014s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.605522806s FAIL (2025-09-23 16:30:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.605556569s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.139750295s FAIL (2025-09-23 16:30:15 UTC) orbiter warning: context deadline exceeded Duration: 6.140000871s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.404073339s FAIL (2025-09-23 16:30:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.40411015s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.130022379s FAIL (2025-09-23 16:29:44 UTC) orbiter warning: context deadline exceeded Duration: 6.130048611s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.957088404s FAIL (2025-09-23 16:29:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 5.957122429s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.981437077s FAIL (2025-09-23 16:29:04 UTC) orbiter warning: context deadline exceeded Duration: 5.981463638s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.99471339s FAIL (2025-09-23 16:28:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.994736575s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.025957502s FAIL (2025-09-23 16:28:22 UTC) orbiter warning: context deadline exceeded Duration: 6.025984505s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 7.262136449s FAIL (2025-09-23 16:27:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 7.262165395s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.339931352s FAIL (2025-09-23 16:27:19 UTC) orbiter warning: context deadline exceeded Duration: 6.339961411s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.175042747s FAIL (2025-09-23 16:26:47 UTC) orbiter warning: context deadline exceeded Duration: 6.175068767s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.258848967s FAIL (2025-09-23 16:26:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.258884895s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.032133197s FAIL (2025-09-23 16:26:19 UTC) orbiter warning: context deadline exceeded Duration: 6.032160308s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.104725291s FAIL (2025-09-23 16:26:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.104757081s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.363308072s FAIL (2025-09-23 16:25:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.363345002s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.396611588s FAIL (2025-09-23 16:25:14 UTC) orbiter warning: context deadline exceeded Duration: 6.396643018s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 24.678677887s FAIL (2025-09-23 16:24:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 24.678717263s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.140425008s FAIL (2025-09-23 16:24:14 UTC) orbiter warning: context deadline exceeded Duration: 6.140454815s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.115873887s FAIL (2025-09-23 16:23:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.115908603s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.648996526s FAIL (2025-09-23 16:23:04 UTC) orbiter warning: context deadline exceeded Duration: 6.64901912s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.150509742s FAIL (2025-09-23 16:22:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.150542996s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.505735402s FAIL (2025-09-23 16:22:04 UTC) orbiter warning: context deadline exceeded Duration: 6.505762515s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 7.040011382s FAIL (2025-09-23 16:21:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 7.040045647s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.304514423s FAIL (2025-09-23 16:21:45 UTC) orbiter warning: context deadline exceeded Duration: 6.30454363s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.964119762s FAIL (2025-09-23 16:21:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 5.964150209s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.699749506s FAIL (2025-09-23 16:20:31 UTC) orbiter warning: context deadline exceeded Duration: 5.699776367s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 13.582201534s FAIL (2025-09-23 16:20:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 13.582264333s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001646849s FAIL (2025-09-23 16:20:13 UTC) orbiter warning: context deadline exceeded Duration: 30.00167347s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.265347973s FAIL (2025-09-23 16:19:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.265380025s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.758576624s FAIL (2025-09-23 16:18:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.758623715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.180653139s FAIL (2025-09-23 16:18:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.180686142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.947950446s FAIL (2025-09-23 16:17:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.947990231s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.039650999s FAIL (2025-09-23 16:16:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.039707025s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.09967635s FAIL (2025-09-23 16:15:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.099709533s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.353739447s FAIL (2025-09-23 16:14:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.353784573s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.277727837s FAIL (2025-09-23 16:13:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.277761682s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.084270022s FAIL (2025-09-23 16:12:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.084302113s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.13104663s FAIL (2025-09-23 16:12:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.131077208s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.077712032s FAIL (2025-09-23 16:11:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.077744624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.021144308s FAIL (2025-09-23 16:10:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.021178364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.149060593s FAIL (2025-09-23 16:09:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 6.149180803s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.357678966s FAIL (2025-09-23 16:09:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 7.357710386s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002727552s FAIL (2025-09-23 16:09:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, Piece ID: Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA) context deadline exceeded; piecestore: (Node ID: 12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, Piece ID: ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA) context deadline exceeded ======LOGS====== Sep 23 16:08:38.605894181 (2h37m24.067986664s) [upload=2325]: Starting upload Sep 23 16:08:38.606004200 (2h37m24.068096693s) [upload=2325]: Done starting upload Sep 23 16:08:38.606176218 (2h37m24.068268701s) [upload=2325]: Uploading segments... Sep 23 16:08:38.606277210 (2h37m24.068369683s) [upload=2325]: Got next segment. Inline: false Sep 23 16:08:38.606318859 (2h37m24.068411342s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 23 16:08:38.606324821 (2h37m24.068417304s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 23 16:08:38.852132441 (2h37m24.314224925s) [seg_pos={0 0}, upload=2325]: Begin upload segment... Sep 23 16:08:38.852225929 (2h37m24.314318402s) [seg_pos={0 0}, upload=2325]: Done begin upload segment. Sep 23 16:08:38.852268150 (2h37m24.314360623s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.852333363 (2h37m24.314425866s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853073563 (2h37m24.315166037s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853408290 (2h37m24.315500763s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853673696 (2h37m24.315766179s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.853967866 (2h37m24.316060349s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854047137 (2h37m24.31613961s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854304707 (2h37m24.31639718s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854435016 (2h37m24.316527499s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854553952 (2h37m24.316646425s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854819237 (2h37m24.31691172s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.854858622 (2h37m24.316951095s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855251851 (2h37m24.317344324s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.855651973 (2h37m24.317744456s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856044019 (2h37m24.318136502s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:38.856461354 (2h37m24.318553827s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Uploading piece... Sep 23 16:08:40.657981922 (2h37m26.120074406s) [upload=2325]: Next returned nil segment Sep 23 16:08:40.658017650 (2h37m26.120110144s) [upload=2325]: Waiting for error group managing segments... Sep 23 16:08:40.965574730 (2h37m26.427667203s) [noise=true , address=45.91.116.5:4298 , piece_id=4T5H6GEBD5CWFHF7WKB4ISNKLF2ZS5DFVRZFWMQR7JUMWD6OFQBA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.017924353 (2h37m26.480016836s) [noise=true , address=45.91.116.9:4262 , piece_id=LO2THA7C52KXZGEPEFXDSJOUBQDUIAJCO6UNQF7ZXZ7ETHN4KIOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.055061418 (2h37m26.517153891s) [noise=true , address=45.91.116.12:4225 , piece_id=NF6W3OWZ7CD2XYLUPWAQX6EPGI753JCTD6GYGX6PSIIUQ5OD6SOA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.059138873 (2h37m26.521231346s) [noise=true , address=77.90.31.10:4519 , piece_id=V7GQEGTIC643FJXEIXRRX64APJQD4YKJFLKEHXFBWXMRUGQ6I4KA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.068239702 (2h37m26.530332175s) [noise=true , address=45.91.116.11:4117 , piece_id=Q2P4XNRWOQEM257IMW253FIOTAWSPG3IEL55VTB2NHXSAQWAXH2A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.071035325 (2h37m26.533127798s) [noise=true , address=45.91.116.10:4082 , piece_id=YP64RUTIYQWN775JB527KG3KOJQMMIK6AJYRAJLPAIQBQH4NIZVA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.120390436 (2h37m26.582482919s) [noise=true , address=45.91.116.12:4227 , piece_id=IJTDTA3FDZCCVCCMWN3UWSNTWTSDU6UMY6F24GENRLVD3IXJ33WQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:41.262045750 (2h37m26.724138234s) [noise=true , address=77.90.31.12:4589 , piece_id=IRIEBUXYXWV2WHWNQDZT2ZPE75VHH4W3SURAYJ4MKMZVLIUUAUBQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.626787709 (2h37m28.088880182s) [noise=true , address=145.79.227.19:4079 , piece_id=ETJUASKFTZZHS7LTVHCWQDCG6T4VKEQOAUZO6G422UQFFYQRBCTA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:42.641124409 (2h37m28.103216882s) [noise=true , address=145.79.227.15:4876 , piece_id=Q6S22GHTR2N3NF7X63GZ56XTFC52CSYI7QED2XDQYQA3BFJLFWAA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.056725062 (2h37m28.518817545s) [noise=true , address=145.79.227.14:4839 , piece_id=MPMNQZLCBFLMMBPYEQCJRJTTCZGZPZRGPDYW55PBM2I6V5RKXBAQ, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.577366321 (2h37m29.039458804s) [noise=true , address=145.79.227.5:4222 , piece_id=HE27KIBGHAZPFT5FUX7FL4KLMWRG3YBZ74N4Q4FGROSK2VFA5S3A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993428202 (2h37m29.455520675s) [noise=true , address=145.79.227.11:4731 , piece_id=46MQMGAXKX2MFI2IXJL7ZWTBVXYQRM6XA7XD565K2F73AKNSTN6A, seg_pos={0 0}, upload=2325]: Done uploading piece. err: <nil> Sep 23 16:08:43.993452629 (2h37m29.455545102s) [seg_pos={0 0}, upload=2325]: Segment reached optimal threshold of 13 pieces. Sep 23 16:08:43.993624226 (2h37m29.455716709s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993637912 (2h37m29.455730395s) [noise=true , address=145.79.227.6:4259 , piece_id=M3MEYQ2XOMDXINQT2MKK2PPQ5DLFLBJMHVI4PWIRXLKSJK74BDMA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993678629 (2h37m29.455771112s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52350->145.79.227.13:4801: use of closed network connection Sep 23 16:08:43.993693137 (2h37m29.45578561s) [noise=true , address=145.79.227.13:4801 , piece_id=Z3JOWJVUQ5BCG36PVVCXWCULB5PDWR4HJWN7NPQF4VDENEMHPCEA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993722223 (2h37m29.455814706s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: expected piece hash; context canceled; EOF Sep 23 16:08:43.993731370 (2h37m29.455823853s) [noise=true , address=145.79.227.3:4150 , piece_id=A6PSDWLOA3E2OGLJ3PW62BULLD2HAGWALQKCMNAGDURXSB3EUOQA, seg_pos={0 0}, upload=2325]: Long tail cancellation due to optimal threshold Sep 23 16:08:43.993751087 (2h37m29.45584356s) [seg_pos={0 0}, upload=2325]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 23 16:08:43.993849845 (2h37m29.455942328s) [upload=2325]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 23 16:08:43.993858933 (2h37m29.455951416s) [upload=2325]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 23 16:08:44.305789411 (2h37m29.767881884s) [upload=2325]: Done uploading segments... Sep 23 16:08:44.305854725 (2h37m29.767947209s) [upload=2325]: Upload finished. err: <nil> Duration: 30.002750665s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 315.654893ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:34:07 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 315.683079ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 400.890329ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:32:52 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 400.942177ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 263.70855ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:31:39 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 263.778813ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 399.21553ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:30:48 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 399.2472ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 370.146137ms target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-09-23 15:30:03 UTC) orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 207.211.208.129:443: i/o timeout Duration: 370.218194ms edge gateway vhost bgp us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)