Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X.X.....XX.....X........X...........X........X..XXX........] duration: 492.490737ms FAIL (2025-09-22 04:16:23 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 492.516607ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..XXX...X.......X...X.........XX..................X.X...X..X] duration: 30.000780963s FAIL (2025-09-22 04:16:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00083841s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X.X.....XX.....X........X...........X........X..XXX.......] duration: 399.701813ms FAIL (2025-09-22 04:15:38 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 399.724064ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...XXX...X.......X...X.........XX..................X.X...X..] duration: 473.234082ms FAIL (2025-09-22 04:15:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 473.255803ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X.X.....XX.....X........X...........X........X..XXX......] duration: 431.374962ms FAIL (2025-09-22 04:15:13 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 431.40593ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [X...XXX...X.......X...X.........XX..................X.X...X.] duration: 411.183669ms FAIL (2025-09-22 04:15:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 411.225066ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [X...X.X.....XX.....X........X...........X........X..XXX.....] duration: 427.727537ms FAIL (2025-09-22 04:14:55 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 427.755038ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [XX...XXX...X.......X...X.........XX..................X.X...X] duration: 30.000634919s FAIL (2025-09-22 04:14:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000798817s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.X...X.X.....XX.....X........X...........X........X..XXX....] duration: 726.340158ms FAIL (2025-09-22 04:14:09 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 726.363402ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.XX...XXX...X.......X...X.........XX..................X.X...] duration: 743.818496ms FAIL (2025-09-22 04:13:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 743.855495ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..X...X.X.....XX.....X........X...........X........X..XXX...] duration: 756.922847ms FAIL (2025-09-22 04:13:19 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 756.953525ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X.XX...XXX...X.......X...X.........XX..................X.X..] duration: 950.768126ms FAIL (2025-09-22 04:12:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 950.785428ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...X...X.X.....XX.....X........X...........X........X..XXX..] duration: 507.07654ms FAIL (2025-09-22 04:12:07 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 507.097038ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X.XX...XXX...X.......X...X.........XX..................X.X.] duration: 412.997282ms FAIL (2025-09-22 04:11:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 413.007993ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..X.XX...XXX...X.......X...X.........XX..................X.X] duration: 30.000555051s FAIL (2025-09-22 04:11:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000593272s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...X.XX...XXX...X.......X...X.........XX..................X.] duration: 416.163905ms FAIL (2025-09-22 04:11:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 416.366546ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X...X...X.X.....XX.....X........X...........X........X..XXX.] duration: 662.831358ms FAIL (2025-09-22 04:10:54 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 662.850995ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X...X.XX...XXX...X.......X...X.........XX..................X] duration: 30.001278846s FAIL (2025-09-22 04:10:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001296438s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..X...X...X.X.....XX.....X........X...........X........X..XX] duration: 30.014098894s FAIL (2025-09-22 04:09:43 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:09:13.266669203 (566h0m48.619454331s) [upload=498140]: Starting upload Sep 22 04:09:13.266773468 (566h0m48.619558597s) [upload=498140]: Done starting upload Sep 22 04:09:13.266845233 (566h0m48.619630361s) [upload=498140]: Uploading segments... Sep 22 04:09:13.267158310 (566h0m48.619943438s) [upload=498140]: Got next segment. Inline: false Sep 22 04:09:13.267210959 (566h0m48.619996087s) [upload=498140]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:09:13.267216770 (566h0m48.620001898s) [upload=498140]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:09:13.290118577 (566h0m48.642903705s) [seg_pos={0 0}, upload=498140]: Begin upload segment... Sep 22 04:09:13.290174963 (566h0m48.642960091s) [seg_pos={0 0}, upload=498140]: Done begin upload segment. Sep 22 04:09:13.290205761 (566h0m48.642990889s) [noise=true , address=77.90.31.17:4771 , piece_id=OOL4NQYQD3AVW4SQPMQXQRN26ICF5DSTQJVHGPGXDC3FSSSMKRMQ, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.290238042 (566h0m48.643023169s) [noise=true , address=45.91.116.8:4045 , piece_id=XEIOE32NYXHO5WO5IAZIWPHZI2SJLADMSKVSW7Y2T2LEFDC6XSUQ, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.290668859 (566h0m48.643453988s) [noise=true , address=45.91.116.12:4225 , piece_id=B2XDYXSN37XMR5A4X7CLXBYUQCNE5VLXVTASNSR2TVOVZDRPAESQ, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.290765030 (566h0m48.643550168s) [noise=true , address=145.79.227.3:4150 , piece_id=52SN3KHCWRZYUEA4EV2ZPH2W5BJI7ZOJCI3P5PPCK53N55LFTEAA, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.291076054 (566h0m48.643861182s) [noise=true , address=145.79.227.15:4874 , piece_id=4OE52NILMCIIO4TDRHTF4RLOK533W6JHBIDOJI2LBPOGCYACVN6A, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.291158689 (566h0m48.643943817s) [noise=true , address=77.90.31.14:4661 , piece_id=C5NBAQCJCUTA6ZD3JBT4IQ7AILBPS2AIXJF6E5VIXBPWNUHNWLYA, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.291441780 (566h0m48.644226908s) [noise=true , address=45.91.116.6:4974 , piece_id=5EVWNNFPWKWXZSQ6GJ6JTR53WLEDDGI6D7OYELMLXNFX3PAWANQQ, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.291526028 (566h0m48.644311156s) [noise=true , address=77.90.31.15:4699 , piece_id=6VBYB2XMXMRNS6IICQAFLU3NN7ZVNT4RVZIUAFLTREVVEUTTQILQ, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.291817696 (566h0m48.644602825s) [noise=true , address=77.90.31.11:4553 , piece_id=7N6P2532MS5EK7LMT6HSTDFIH6WA5CEN7AXTHUV2WDF7VCEIDKAQ, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.291989628 (566h0m48.644774757s) [noise=true , address=145.79.227.14:4840 , piece_id=SUPIJOKMHX6KVNEXIKIPMOWWGH4KZDM575PT6BSGDAEUQ74ZL2YA, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.292259224 (566h0m48.645044362s) [noise=true , address=145.79.227.2:4115 , piece_id=MZSF6WKSCMQRBIHTW6ZQHCXD3ZJ4LCWPEWPDPQQ36VNMOA2BLLZA, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.292360835 (566h0m48.645145963s) [noise=true , address=145.79.227.6:4259 , piece_id=LIEMKDFIUNEJE3MKJVEPBFK3QVPRTHAJ3RM5DK7DTNBXOGNMYRWQ, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.292643284 (566h0m48.645428423s) [noise=true , address=45.91.116.13:4155 , piece_id=A4KBA3PM5LFHMG4GR74U3K5OPUTT2OLHCUMPFNTO44II6OLQ7XUA, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.292779772 (566h0m48.64556491s) [noise=true , address=77.90.31.19:4843 , piece_id=NR5KWX2QZI4UC3M5GXSUVWTMDKOUVF33DBEUUSDT6VK7ICUP2CSQ, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.293021435 (566h0m48.645806563s) [noise=true , address=77.90.31.10:4520 , piece_id=XNBJO66VQPWULJMLZIQNH44HFGTLDALYCWSKPZM3CNPVYZJ6F44Q, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.293179421 (566h0m48.645964549s) [noise=true , address=145.79.227.19:4077 , piece_id=TCVCA5TUKMTLVOBHTSZQLILG44LG5BHYITALMJKTMIGRIQADNAVA, seg_pos={0 0}, upload=498140]: Uploading piece... Sep 22 04:09:13.417396865 (566h0m48.770182003s) [upload=498140]: Next returned nil segment Sep 22 04:09:13.417416462 (566h0m48.7702016s) [upload=498140]: Waiting for error group managing segments... Sep 22 04:09:13.426150052 (566h0m48.77893519s) [noise=true , address=45.91.116.6:4974 , piece_id=5EVWNNFPWKWXZSQ6GJ6JTR53WLEDDGI6D7OYELMLXNFX3PAWANQQ, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.427306141 (566h0m48.780091269s) [noise=true , address=45.91.116.13:4155 , piece_id=A4KBA3PM5LFHMG4GR74U3K5OPUTT2OLHCUMPFNTO44II6OLQ7XUA, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.427627193 (566h0m48.780412321s) [noise=true , address=45.91.116.8:4045 , piece_id=XEIOE32NYXHO5WO5IAZIWPHZI2SJLADMSKVSW7Y2T2LEFDC6XSUQ, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.427992459 (566h0m48.780777587s) [noise=true , address=45.91.116.12:4225 , piece_id=B2XDYXSN37XMR5A4X7CLXBYUQCNE5VLXVTASNSR2TVOVZDRPAESQ, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.428601431 (566h0m48.781386559s) [noise=true , address=77.90.31.19:4843 , piece_id=NR5KWX2QZI4UC3M5GXSUVWTMDKOUVF33DBEUUSDT6VK7ICUP2CSQ, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.429504195 (566h0m48.782289324s) [noise=true , address=77.90.31.14:4661 , piece_id=C5NBAQCJCUTA6ZD3JBT4IQ7AILBPS2AIXJF6E5VIXBPWNUHNWLYA, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.429699361 (566h0m48.78248449s) [noise=true , address=77.90.31.11:4553 , piece_id=7N6P2532MS5EK7LMT6HSTDFIH6WA5CEN7AXTHUV2WDF7VCEIDKAQ, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.430160236 (566h0m48.782945365s) [noise=true , address=77.90.31.17:4771 , piece_id=OOL4NQYQD3AVW4SQPMQXQRN26ICF5DSTQJVHGPGXDC3FSSSMKRMQ, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.430320548 (566h0m48.783105675s) [noise=true , address=77.90.31.10:4520 , piece_id=XNBJO66VQPWULJMLZIQNH44HFGTLDALYCWSKPZM3CNPVYZJ6F44Q, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:13.430682718 (566h0m48.783467845s) [noise=true , address=77.90.31.15:4699 , piece_id=6VBYB2XMXMRNS6IICQAFLU3NN7ZVNT4RVZIUAFLTREVVEUTTQILQ, seg_pos={0 0}, upload=498140]: Done uploading piece. err: <nil> Sep 22 04:09:43.251073984 (566h1m18.603859092s) [noise=true , address=145.79.227.2:4115 , piece_id=MZSF6WKSCMQRBIHTW6ZQHCXD3ZJ4LCWPEWPDPQQ36VNMOA2BLLZA, seg_pos={0 0}, upload=498140]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: rpc: dial tcp 145.79.227.2:4115: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:43.251107777 (566h1m18.603892876s) [noise=true , address=145.79.227.14:4840 , piece_id=SUPIJOKMHX6KVNEXIKIPMOWWGH4KZDM575PT6BSGDAEUQ74ZL2YA, seg_pos={0 0}, upload=498140]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: rpc: dial tcp 145.79.227.14:4840: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:43.251131001 (566h1m18.603916109s) [noise=true , address=145.79.227.15:4874 , piece_id=4OE52NILMCIIO4TDRHTF4RLOK533W6JHBIDOJI2LBPOGCYACVN6A, seg_pos={0 0}, upload=498140]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: rpc: dial tcp 145.79.227.15:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:43.251146700 (566h1m18.603931808s) [noise=true , address=145.79.227.6:4259 , piece_id=LIEMKDFIUNEJE3MKJVEPBFK3QVPRTHAJ3RM5DK7DTNBXOGNMYRWQ, seg_pos={0 0}, upload=498140]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: rpc: dial tcp 145.79.227.6:4259: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:43.251162249 (566h1m18.603947357s) [noise=true , address=145.79.227.19:4077 , piece_id=TCVCA5TUKMTLVOBHTSZQLILG44LG5BHYITALMJKTMIGRIQADNAVA, seg_pos={0 0}, upload=498140]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:43.251175655 (566h1m18.603960752s) [noise=true , address=145.79.227.3:4150 , piece_id=52SN3KHCWRZYUEA4EV2ZPH2W5BJI7ZOJCI3P5PPCK53N55LFTEAA, seg_pos={0 0}, upload=498140]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: rpc: dial tcp 145.79.227.3:4150: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:43.251188609 (566h1m18.603973717s) [seg_pos={0 0}, upload=498140]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:09:43.264081021 (566h1m18.616866129s) [upload=498140]: Done uploading segments... Sep 22 04:09:43.264097193 (566h1m18.616882301s) [upload=498140]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.014120706s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [X..X...X...X.X.....XX.....X........X...........X........X..X] duration: 30.014479524s FAIL (2025-09-22 04:09:12 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:08:42.724687795 (566h0m18.077472893s) [upload=498132]: Starting upload Sep 22 04:08:42.724738921 (566h0m18.077524019s) [upload=498132]: Done starting upload Sep 22 04:08:42.724866951 (566h0m18.077652079s) [upload=498132]: Uploading segments... Sep 22 04:08:42.725011082 (566h0m18.077796221s) [upload=498132]: Got next segment. Inline: false Sep 22 04:08:42.725050577 (566h0m18.077835705s) [upload=498132]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:08:42.725056107 (566h0m18.077841235s) [upload=498132]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:08:42.745692443 (566h0m18.098477571s) [seg_pos={0 0}, upload=498132]: Begin upload segment... Sep 22 04:08:42.745731165 (566h0m18.098516303s) [seg_pos={0 0}, upload=498132]: Done begin upload segment. Sep 22 04:08:42.745825963 (566h0m18.098611091s) [noise=true , address=77.90.31.14:4661 , piece_id=SKTJNFXEJMKBMOQDZT6PAFERHYQMGS4P4VEXGSGWP4EBYU6UL7SQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.745850669 (566h0m18.098635848s) [noise=true , address=145.79.227.13:4801 , piece_id=2T6JBNVZI7LJSVRQSVXGRJXPBBKFVXWB7ZWPNP7EUJBG3UHGVC5A, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.746470503 (566h0m18.099255631s) [noise=true , address=45.91.116.13:4155 , piece_id=QBVNMFRUSRKDLFR24KHXDRAS2APORPV4B6ZGVI5MAO7VJ5XJHQBQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.746950874 (566h0m18.099736002s) [noise=true , address=77.90.31.18:4806 , piece_id=URWJPZGJURWUONQGFCNET6AUSDK7HXIQJP7I5ON43KEB2OWSHACQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.747041995 (566h0m18.099827123s) [noise=true , address=45.91.116.14:4191 , piece_id=LY5FEXOJG3RFPSWY2TLB3SN2CRMMFPHXSG6KK2MYR2XORGMECH2Q, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.747504153 (566h0m18.10028928s) [noise=true , address=77.90.31.17:4770 , piece_id=ALDXFYNSKRRCLLB6N3BXOUHHVWTCS74GUGJGR2J4E7EWQ3J566QA, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.748027364 (566h0m18.100812502s) [noise=true , address=145.79.227.12:4767 , piece_id=O7TATBYEFXCFINHXUAR7RLGSFCMB2TQDFC3UC2BN6Q75YQ63P44Q, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.748066918 (566h0m18.100852056s) [noise=true , address=77.90.31.19:4842 , piece_id=2KVVKX57UNAK66DOUGJ632AHLFMV7CG4MAJC7RQG5ZHYMXUATMOQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.750174684 (566h0m18.102959812s) [noise=true , address=45.91.116.12:4226 , piece_id=KUHXGJ5YATUWIL22TTDAZQZU6B4BMS44KUKQIPMJBGAAX4J6JVEA, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.750799727 (566h0m18.103584855s) [noise=true , address=145.79.227.11:4731 , piece_id=TJB3XP6EIY34PEU2OKNLI3JAIEY43BEKHKHG3MBOMOQCQVUG6XZQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.751307710 (566h0m18.104092838s) [noise=true , address=77.90.31.14:4662 , piece_id=C3HFUICGQOUCIBQEDIICZJ4TAKJFJTP4MWBUO6BCJJAECA2YX4VQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.751693594 (566h0m18.104478722s) [noise=true , address=145.79.227.16:4910 , piece_id=MCTOWLNZGYY5C6UOJ5BTXBNSCEIK4M2XPCBGXQXXMJCTVBMRTKXQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.752653976 (566h0m18.105439115s) [noise=true , address=145.79.227.19:4077 , piece_id=IBAMVUXSJV6PYG3YA5Q4XBZMLIOKNTOX6UZAGYW33UL4O57I6SPA, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.756295881 (566h0m18.109081009s) [noise=true , address=145.79.227.2:4113 , piece_id=AMYUF4W5U5CM6GZKQTTHURWODFSEBRWQQ5NM7ILKOJ2D3CGBJHYQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.757463020 (566h0m18.110248159s) [noise=true , address=77.90.31.13:4627 , piece_id=EXWWLSQVYCE2RHWJQSXP3GDUNCE5JUWUKHV2DEMRA3TGIBHGU2SA, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.759532214 (566h0m18.112317343s) [noise=true , address=77.90.31.10:4517 , piece_id=KC45SW4EGKTRAHKJE6Q6NENMDKQXNUQ3ZLE3ETVNEL3WAUOBJGCQ, seg_pos={0 0}, upload=498132]: Uploading piece... Sep 22 04:08:42.883268695 (566h0m18.236053803s) [upload=498132]: Next returned nil segment Sep 22 04:08:42.883278784 (566h0m18.236063892s) [upload=498132]: Waiting for error group managing segments... Sep 22 04:08:42.892397666 (566h0m18.245182755s) [noise=true , address=45.91.116.12:4226 , piece_id=KUHXGJ5YATUWIL22TTDAZQZU6B4BMS44KUKQIPMJBGAAX4J6JVEA, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.893040373 (566h0m18.245825471s) [noise=true , address=45.91.116.14:4191 , piece_id=LY5FEXOJG3RFPSWY2TLB3SN2CRMMFPHXSG6KK2MYR2XORGMECH2Q, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.893372215 (566h0m18.246157314s) [noise=true , address=77.90.31.19:4842 , piece_id=2KVVKX57UNAK66DOUGJ632AHLFMV7CG4MAJC7RQG5ZHYMXUATMOQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.893816249 (566h0m18.246601347s) [noise=true , address=77.90.31.14:4661 , piece_id=SKTJNFXEJMKBMOQDZT6PAFERHYQMGS4P4VEXGSGWP4EBYU6UL7SQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.894240635 (566h0m18.247025743s) [noise=true , address=77.90.31.18:4806 , piece_id=URWJPZGJURWUONQGFCNET6AUSDK7HXIQJP7I5ON43KEB2OWSHACQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.894598267 (566h0m18.247383375s) [noise=true , address=77.90.31.13:4627 , piece_id=EXWWLSQVYCE2RHWJQSXP3GDUNCE5JUWUKHV2DEMRA3TGIBHGU2SA, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.894963312 (566h0m18.24774842s) [noise=true , address=77.90.31.14:4662 , piece_id=C3HFUICGQOUCIBQEDIICZJ4TAKJFJTP4MWBUO6BCJJAECA2YX4VQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.896237963 (566h0m18.249023061s) [noise=true , address=77.90.31.17:4770 , piece_id=ALDXFYNSKRRCLLB6N3BXOUHHVWTCS74GUGJGR2J4E7EWQ3J566QA, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.896286264 (566h0m18.249071361s) [noise=true , address=77.90.31.10:4517 , piece_id=KC45SW4EGKTRAHKJE6Q6NENMDKQXNUQ3ZLE3ETVNEL3WAUOBJGCQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:42.908808744 (566h0m18.261593842s) [noise=true , address=45.91.116.13:4155 , piece_id=QBVNMFRUSRKDLFR24KHXDRAS2APORPV4B6ZGVI5MAO7VJ5XJHQBQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: <nil> Sep 22 04:08:51.028678798 (566h0m26.381463896s) [noise=true , address=145.79.227.11:4731 , piece_id=TJB3XP6EIY34PEU2OKNLI3JAIEY43BEKHKHG3MBOMOQCQVUG6XZQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:12.710527414 (566h0m48.063312543s) [noise=true , address=145.79.227.13:4801 , piece_id=2T6JBNVZI7LJSVRQSVXGRJXPBBKFVXWB7ZWPNP7EUJBG3UHGVC5A, seg_pos={0 0}, upload=498132]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:12.710625278 (566h0m48.063410406s) [noise=true , address=145.79.227.12:4767 , piece_id=O7TATBYEFXCFINHXUAR7RLGSFCMB2TQDFC3UC2BN6Q75YQ63P44Q, seg_pos={0 0}, upload=498132]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: rpc: dial tcp 145.79.227.12:4767: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:12.710686633 (566h0m48.063471761s) [noise=true , address=145.79.227.16:4910 , piece_id=MCTOWLNZGYY5C6UOJ5BTXBNSCEIK4M2XPCBGXQXXMJCTVBMRTKXQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:12.710729835 (566h0m48.063514964s) [noise=true , address=145.79.227.2:4113 , piece_id=AMYUF4W5U5CM6GZKQTTHURWODFSEBRWQQ5NM7ILKOJ2D3CGBJHYQ, seg_pos={0 0}, upload=498132]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:12.710849239 (566h0m48.063634377s) [noise=true , address=145.79.227.19:4077 , piece_id=IBAMVUXSJV6PYG3YA5Q4XBZMLIOKNTOX6UZAGYW33UL4O57I6SPA, seg_pos={0 0}, upload=498132]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: rpc: dial tcp 145.79.227.19:4077: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:09:12.710891438 (566h0m48.063676566s) [seg_pos={0 0}, upload=498132]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:09:12.723251302 (566h0m48.07603643s) [upload=498132]: Done uploading segments... Sep 22 04:09:12.723279566 (566h0m48.076064693s) [upload=498132]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.014512577s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.X...X.XX...XXX...X.......X...X.........XX..................] duration: 580.23023ms FAIL (2025-09-22 04:08:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 580.260236ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..X...X.XX...XXX...X.......X...X.........XX.................] duration: 902.395972ms FAIL (2025-09-22 04:07:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 902.435176ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X..X...X...X.X.....XX.....X........X...........X........X..] duration: 522.874347ms FAIL (2025-09-22 04:07:40 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 04:06:09.636297061 (565h57m44.98908216s) [upload=498094]: Starting upload Sep 22 04:06:09.636349811 (565h57m44.989134908s) [upload=498094]: Done starting upload Sep 22 04:06:09.636406928 (565h57m44.989192026s) [upload=498094]: Uploading segments... Sep 22 04:06:09.636588959 (565h57m44.989374057s) [upload=498094]: Got next segment. Inline: false Sep 22 04:06:09.636653320 (565h57m44.989438428s) [upload=498094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:06:09.636659281 (565h57m44.989444379s) [upload=498094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:06:09.751508215 (565h57m45.104293313s) [seg_pos={0 0}, upload=498094]: Begin upload segment... Sep 22 04:06:09.751545334 (565h57m45.104330433s) [seg_pos={0 0}, upload=498094]: Done begin upload segment. Sep 22 04:06:09.751572165 (565h57m45.104357273s) [noise=true , address=45.91.116.9:4262 , piece_id=SF76QXCSOSOFFOSQSCQPCHSE42RSGBSW7UA55CFSQV3IJ6SEVUSA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.751607000 (565h57m45.104392098s) [noise=true , address=45.91.116.10:4081 , piece_id=ZSXHNSUI7WP6MOI4CZKP64J3ZA2S2IOW2NCNL4IUYJRBF6K2PBCA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.751962167 (565h57m45.104747264s) [noise=true , address=145.79.227.11:4729 , piece_id=DPZMQNMF2KVN5GFDCFUTXGGFDS3RKUAFEKUIUP6JPOOXUE3465PQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752048008 (565h57m45.104833105s) [noise=true , address=145.79.227.11:4730 , piece_id=MMMQHMZZTM2SDMZSVACKUY3PW2KIZ5QK4L4AF3LF32JGTNITLBXQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752266387 (565h57m45.105051505s) [noise=true , address=45.91.116.10:4082 , piece_id=CPA2ISXBRP6PF4JVYYGGY3BYF3DFVFN74NF75NIJTQ7CQIUXUXOQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752350636 (565h57m45.105135734s) [noise=true , address=45.91.116.11:4118 , piece_id=3YG4ECFKLATKL4RL3NP4LFV5ZY7C6ERCYAHZDHTCIEFF7FKRS6PQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752533359 (565h57m45.105318456s) [noise=true , address=145.79.227.7:4295 , piece_id=LKOIQOEUD6SW3KMEKQ4357LKQO4YPOZOAR4Y7VCBCVFULEHB3B5A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752613489 (565h57m45.105398587s) [noise=true , address=45.91.116.13:4154 , piece_id=OZWZJGP3ILRFWQ4UZZBYZGWDLBGQVKCWGP6DSRJPNLYYXTM7HE7Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752802694 (565h57m45.105587792s) [noise=true , address=77.90.31.16:4733 , piece_id=HPGXY6LNBBWEITQKIRSS5FUWN27ANT53SXPBDB2OVQ2V7NOGWG5Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752890639 (565h57m45.105675747s) [noise=true , address=77.90.31.17:4771 , piece_id=5F5TF72GWADPW4VY7T27HFLSZMBPTQRLNBFE2N26YUSQ7Y43KXVQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753138754 (565h57m45.105923852s) [noise=true , address=77.90.31.10:4520 , piece_id=A6IPQ4HSYUAEQFDJIF2524SFLMBEUCAMYJD5MNVMPOL4JHCXJPGQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753237640 (565h57m45.106022748s) [noise=true , address=145.79.227.13:4801 , piece_id=7D7G5GGMZXFOPQ5LSL73VK6MI3VJFPFJUYV5VCN2HBEIZTH4BE5Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753459406 (565h57m45.106244504s) [noise=true , address=145.79.227.2:4113 , piece_id=YFDZFNYX6XNZAIQJFCZNZO3WCUBL6UBQBVIO3J4SW5O4A2DKICZQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753541541 (565h57m45.106326639s) [noise=true , address=145.79.227.15:4873 , piece_id=I3FOTPN4FUGXFSNLOV6F55HE5SK6NACGQTVEIUPL2ZEG6I3QKK2A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753773366 (565h57m45.106558463s) [noise=true , address=45.91.116.11:4117 , piece_id=2DTFQXZBN2A2PKBLP5VY7A7JR226OCXUEEN5WMSNSUVZ6EL3MI2A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.754075913 (565h57m45.106861011s) [noise=true , address=145.79.227.4:4185 , piece_id=OV6KM67YXWDINQZVE6WKVXVP35P5XAXGDO4WKT2YWBQQINFDFDRA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.835461850 (565h57m45.188246959s) [upload=498094]: Next returned nil segment Sep 22 04:06:09.835473382 (565h57m45.18825849s) [upload=498094]: Waiting for error group managing segments... Sep 22 04:06:09.844018979 (565h57m45.196804077s) [noise=true , address=45.91.116.10:4082 , piece_id=CPA2ISXBRP6PF4JVYYGGY3BYF3DFVFN74NF75NIJTQ7CQIUXUXOQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.844981535 (565h57m45.197766633s) [noise=true , address=45.91.116.11:4117 , piece_id=2DTFQXZBN2A2PKBLP5VY7A7JR226OCXUEEN5WMSNSUVZ6EL3MI2A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845349495 (565h57m45.198134593s) [noise=true , address=45.91.116.10:4081 , piece_id=ZSXHNSUI7WP6MOI4CZKP64J3ZA2S2IOW2NCNL4IUYJRBF6K2PBCA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845438762 (565h57m45.19822387s) [noise=true , address=45.91.116.11:4118 , piece_id=3YG4ECFKLATKL4RL3NP4LFV5ZY7C6ERCYAHZDHTCIEFF7FKRS6PQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845704431 (565h57m45.19848953s) [noise=true , address=45.91.116.13:4154 , piece_id=OZWZJGP3ILRFWQ4UZZBYZGWDLBGQVKCWGP6DSRJPNLYYXTM7HE7Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846133176 (565h57m45.198918274s) [noise=true , address=77.90.31.16:4733 , piece_id=HPGXY6LNBBWEITQKIRSS5FUWN27ANT53SXPBDB2OVQ2V7NOGWG5Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846306451 (565h57m45.199091559s) [noise=true , address=45.91.116.9:4262 , piece_id=SF76QXCSOSOFFOSQSCQPCHSE42RSGBSW7UA55CFSQV3IJ6SEVUSA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846642913 (565h57m45.199428021s) [noise=true , address=77.90.31.10:4520 , piece_id=A6IPQ4HSYUAEQFDJIF2524SFLMBEUCAMYJD5MNVMPOL4JHCXJPGQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846727091 (565h57m45.199512189s) [noise=true , address=77.90.31.17:4771 , piece_id=5F5TF72GWADPW4VY7T27HFLSZMBPTQRLNBFE2N26YUSQ7Y43KXVQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:11.254886917 (565h57m46.607672015s) [noise=true , address=145.79.227.11:4730 , piece_id=MMMQHMZZTM2SDMZSVACKUY3PW2KIZ5QK4L4AF3LF32JGTNITLBXQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:11.254997294 (565h57m46.607782392s) [noise=true , address=145.79.227.11:4729 , piece_id=DPZMQNMF2KVN5GFDCFUTXGGFDS3RKUAFEKUIUP6JPOOXUE3465PQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620098065 (565h58m14.972883173s) [noise=true , address=145.79.227.15:4873 , piece_id=I3FOTPN4FUGXFSNLOV6F55HE5SK6NACGQTVEIUPL2ZEG6I3QKK2A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620146045 (565h58m14.972931154s) [noise=true , address=145.79.227.4:4185 , piece_id=OV6KM67YXWDINQZVE6WKVXVP35P5XAXGDO4WKT2YWBQQINFDFDRA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620187432 (565h58m14.972972541s) [noise=true , address=145.79.227.13:4801 , piece_id=7D7G5GGMZXFOPQ5LSL73VK6MI3VJFPFJUYV5VCN2HBEIZTH4BE5Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620225484 (565h58m14.973010583s) [noise=true , address=145.79.227.7:4295 , piece_id=LKOIQOEUD6SW3KMEKQ4357LKQO4YPOZOAR4Y7VCBCVFULEHB3B5A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: rpc: dial tcp 145.79.227.7:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620269477 (565h58m14.973054585s) [noise=true , address=145.79.227.2:4113 , piece_id=YFDZFNYX6XNZAIQJFCZNZO3WCUBL6UBQBVIO3J4SW5O4A2DKICZQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; context deadline exceeded Sep 22 04:06:39.620307199 (565h58m14.973092307s) [seg_pos={0 0}, upload=498094]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 04:06:39.637082920 (565h58m14.989868028s) [upload=498094]: Done uploading segments... Sep 22 04:06:39.637102205 (565h58m14.989887314s) [upload=498094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 522.901107ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...X...X.XX...XXX...X.......X...X.........XX................] duration: 435.631919ms FAIL (2025-09-22 04:06:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 435.658388ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..X..X...X...X.X.....XX.....X........X...........X........X.] duration: 374.90336ms FAIL (2025-09-22 04:06:41 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 04:06:09.636297061 (565h57m44.98908216s) [upload=498094]: Starting upload Sep 22 04:06:09.636349811 (565h57m44.989134908s) [upload=498094]: Done starting upload Sep 22 04:06:09.636406928 (565h57m44.989192026s) [upload=498094]: Uploading segments... Sep 22 04:06:09.636588959 (565h57m44.989374057s) [upload=498094]: Got next segment. Inline: false Sep 22 04:06:09.636653320 (565h57m44.989438428s) [upload=498094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:06:09.636659281 (565h57m44.989444379s) [upload=498094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:06:09.751508215 (565h57m45.104293313s) [seg_pos={0 0}, upload=498094]: Begin upload segment... Sep 22 04:06:09.751545334 (565h57m45.104330433s) [seg_pos={0 0}, upload=498094]: Done begin upload segment. Sep 22 04:06:09.751572165 (565h57m45.104357273s) [noise=true , address=45.91.116.9:4262 , piece_id=SF76QXCSOSOFFOSQSCQPCHSE42RSGBSW7UA55CFSQV3IJ6SEVUSA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.751607000 (565h57m45.104392098s) [noise=true , address=45.91.116.10:4081 , piece_id=ZSXHNSUI7WP6MOI4CZKP64J3ZA2S2IOW2NCNL4IUYJRBF6K2PBCA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.751962167 (565h57m45.104747264s) [noise=true , address=145.79.227.11:4729 , piece_id=DPZMQNMF2KVN5GFDCFUTXGGFDS3RKUAFEKUIUP6JPOOXUE3465PQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752048008 (565h57m45.104833105s) [noise=true , address=145.79.227.11:4730 , piece_id=MMMQHMZZTM2SDMZSVACKUY3PW2KIZ5QK4L4AF3LF32JGTNITLBXQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752266387 (565h57m45.105051505s) [noise=true , address=45.91.116.10:4082 , piece_id=CPA2ISXBRP6PF4JVYYGGY3BYF3DFVFN74NF75NIJTQ7CQIUXUXOQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752350636 (565h57m45.105135734s) [noise=true , address=45.91.116.11:4118 , piece_id=3YG4ECFKLATKL4RL3NP4LFV5ZY7C6ERCYAHZDHTCIEFF7FKRS6PQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752533359 (565h57m45.105318456s) [noise=true , address=145.79.227.7:4295 , piece_id=LKOIQOEUD6SW3KMEKQ4357LKQO4YPOZOAR4Y7VCBCVFULEHB3B5A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752613489 (565h57m45.105398587s) [noise=true , address=45.91.116.13:4154 , piece_id=OZWZJGP3ILRFWQ4UZZBYZGWDLBGQVKCWGP6DSRJPNLYYXTM7HE7Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752802694 (565h57m45.105587792s) [noise=true , address=77.90.31.16:4733 , piece_id=HPGXY6LNBBWEITQKIRSS5FUWN27ANT53SXPBDB2OVQ2V7NOGWG5Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752890639 (565h57m45.105675747s) [noise=true , address=77.90.31.17:4771 , piece_id=5F5TF72GWADPW4VY7T27HFLSZMBPTQRLNBFE2N26YUSQ7Y43KXVQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753138754 (565h57m45.105923852s) [noise=true , address=77.90.31.10:4520 , piece_id=A6IPQ4HSYUAEQFDJIF2524SFLMBEUCAMYJD5MNVMPOL4JHCXJPGQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753237640 (565h57m45.106022748s) [noise=true , address=145.79.227.13:4801 , piece_id=7D7G5GGMZXFOPQ5LSL73VK6MI3VJFPFJUYV5VCN2HBEIZTH4BE5Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753459406 (565h57m45.106244504s) [noise=true , address=145.79.227.2:4113 , piece_id=YFDZFNYX6XNZAIQJFCZNZO3WCUBL6UBQBVIO3J4SW5O4A2DKICZQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753541541 (565h57m45.106326639s) [noise=true , address=145.79.227.15:4873 , piece_id=I3FOTPN4FUGXFSNLOV6F55HE5SK6NACGQTVEIUPL2ZEG6I3QKK2A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753773366 (565h57m45.106558463s) [noise=true , address=45.91.116.11:4117 , piece_id=2DTFQXZBN2A2PKBLP5VY7A7JR226OCXUEEN5WMSNSUVZ6EL3MI2A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.754075913 (565h57m45.106861011s) [noise=true , address=145.79.227.4:4185 , piece_id=OV6KM67YXWDINQZVE6WKVXVP35P5XAXGDO4WKT2YWBQQINFDFDRA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.835461850 (565h57m45.188246959s) [upload=498094]: Next returned nil segment Sep 22 04:06:09.835473382 (565h57m45.18825849s) [upload=498094]: Waiting for error group managing segments... Sep 22 04:06:09.844018979 (565h57m45.196804077s) [noise=true , address=45.91.116.10:4082 , piece_id=CPA2ISXBRP6PF4JVYYGGY3BYF3DFVFN74NF75NIJTQ7CQIUXUXOQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.844981535 (565h57m45.197766633s) [noise=true , address=45.91.116.11:4117 , piece_id=2DTFQXZBN2A2PKBLP5VY7A7JR226OCXUEEN5WMSNSUVZ6EL3MI2A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845349495 (565h57m45.198134593s) [noise=true , address=45.91.116.10:4081 , piece_id=ZSXHNSUI7WP6MOI4CZKP64J3ZA2S2IOW2NCNL4IUYJRBF6K2PBCA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845438762 (565h57m45.19822387s) [noise=true , address=45.91.116.11:4118 , piece_id=3YG4ECFKLATKL4RL3NP4LFV5ZY7C6ERCYAHZDHTCIEFF7FKRS6PQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845704431 (565h57m45.19848953s) [noise=true , address=45.91.116.13:4154 , piece_id=OZWZJGP3ILRFWQ4UZZBYZGWDLBGQVKCWGP6DSRJPNLYYXTM7HE7Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846133176 (565h57m45.198918274s) [noise=true , address=77.90.31.16:4733 , piece_id=HPGXY6LNBBWEITQKIRSS5FUWN27ANT53SXPBDB2OVQ2V7NOGWG5Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846306451 (565h57m45.199091559s) [noise=true , address=45.91.116.9:4262 , piece_id=SF76QXCSOSOFFOSQSCQPCHSE42RSGBSW7UA55CFSQV3IJ6SEVUSA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846642913 (565h57m45.199428021s) [noise=true , address=77.90.31.10:4520 , piece_id=A6IPQ4HSYUAEQFDJIF2524SFLMBEUCAMYJD5MNVMPOL4JHCXJPGQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846727091 (565h57m45.199512189s) [noise=true , address=77.90.31.17:4771 , piece_id=5F5TF72GWADPW4VY7T27HFLSZMBPTQRLNBFE2N26YUSQ7Y43KXVQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:11.254886917 (565h57m46.607672015s) [noise=true , address=145.79.227.11:4730 , piece_id=MMMQHMZZTM2SDMZSVACKUY3PW2KIZ5QK4L4AF3LF32JGTNITLBXQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:11.254997294 (565h57m46.607782392s) [noise=true , address=145.79.227.11:4729 , piece_id=DPZMQNMF2KVN5GFDCFUTXGGFDS3RKUAFEKUIUP6JPOOXUE3465PQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620098065 (565h58m14.972883173s) [noise=true , address=145.79.227.15:4873 , piece_id=I3FOTPN4FUGXFSNLOV6F55HE5SK6NACGQTVEIUPL2ZEG6I3QKK2A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620146045 (565h58m14.972931154s) [noise=true , address=145.79.227.4:4185 , piece_id=OV6KM67YXWDINQZVE6WKVXVP35P5XAXGDO4WKT2YWBQQINFDFDRA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620187432 (565h58m14.972972541s) [noise=true , address=145.79.227.13:4801 , piece_id=7D7G5GGMZXFOPQ5LSL73VK6MI3VJFPFJUYV5VCN2HBEIZTH4BE5Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620225484 (565h58m14.973010583s) [noise=true , address=145.79.227.7:4295 , piece_id=LKOIQOEUD6SW3KMEKQ4357LKQO4YPOZOAR4Y7VCBCVFULEHB3B5A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: rpc: dial tcp 145.79.227.7:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620269477 (565h58m14.973054585s) [noise=true , address=145.79.227.2:4113 , piece_id=YFDZFNYX6XNZAIQJFCZNZO3WCUBL6UBQBVIO3J4SW5O4A2DKICZQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; context deadline exceeded Sep 22 04:06:39.620307199 (565h58m14.973092307s) [seg_pos={0 0}, upload=498094]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 04:06:39.637082920 (565h58m14.989868028s) [upload=498094]: Done uploading segments... Sep 22 04:06:39.637102205 (565h58m14.989887314s) [upload=498094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 374.921693ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...X..X...X...X.X.....XX.....X........X...........X........X] duration: 30.019217466s FAIL (2025-09-22 04:06:39 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 22 04:06:09.636297061 (565h57m44.98908216s) [upload=498094]: Starting upload Sep 22 04:06:09.636349811 (565h57m44.989134908s) [upload=498094]: Done starting upload Sep 22 04:06:09.636406928 (565h57m44.989192026s) [upload=498094]: Uploading segments... Sep 22 04:06:09.636588959 (565h57m44.989374057s) [upload=498094]: Got next segment. Inline: false Sep 22 04:06:09.636653320 (565h57m44.989438428s) [upload=498094]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:06:09.636659281 (565h57m44.989444379s) [upload=498094]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:06:09.751508215 (565h57m45.104293313s) [seg_pos={0 0}, upload=498094]: Begin upload segment... Sep 22 04:06:09.751545334 (565h57m45.104330433s) [seg_pos={0 0}, upload=498094]: Done begin upload segment. Sep 22 04:06:09.751572165 (565h57m45.104357273s) [noise=true , address=45.91.116.9:4262 , piece_id=SF76QXCSOSOFFOSQSCQPCHSE42RSGBSW7UA55CFSQV3IJ6SEVUSA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.751607000 (565h57m45.104392098s) [noise=true , address=45.91.116.10:4081 , piece_id=ZSXHNSUI7WP6MOI4CZKP64J3ZA2S2IOW2NCNL4IUYJRBF6K2PBCA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.751962167 (565h57m45.104747264s) [noise=true , address=145.79.227.11:4729 , piece_id=DPZMQNMF2KVN5GFDCFUTXGGFDS3RKUAFEKUIUP6JPOOXUE3465PQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752048008 (565h57m45.104833105s) [noise=true , address=145.79.227.11:4730 , piece_id=MMMQHMZZTM2SDMZSVACKUY3PW2KIZ5QK4L4AF3LF32JGTNITLBXQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752266387 (565h57m45.105051505s) [noise=true , address=45.91.116.10:4082 , piece_id=CPA2ISXBRP6PF4JVYYGGY3BYF3DFVFN74NF75NIJTQ7CQIUXUXOQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752350636 (565h57m45.105135734s) [noise=true , address=45.91.116.11:4118 , piece_id=3YG4ECFKLATKL4RL3NP4LFV5ZY7C6ERCYAHZDHTCIEFF7FKRS6PQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752533359 (565h57m45.105318456s) [noise=true , address=145.79.227.7:4295 , piece_id=LKOIQOEUD6SW3KMEKQ4357LKQO4YPOZOAR4Y7VCBCVFULEHB3B5A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752613489 (565h57m45.105398587s) [noise=true , address=45.91.116.13:4154 , piece_id=OZWZJGP3ILRFWQ4UZZBYZGWDLBGQVKCWGP6DSRJPNLYYXTM7HE7Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752802694 (565h57m45.105587792s) [noise=true , address=77.90.31.16:4733 , piece_id=HPGXY6LNBBWEITQKIRSS5FUWN27ANT53SXPBDB2OVQ2V7NOGWG5Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.752890639 (565h57m45.105675747s) [noise=true , address=77.90.31.17:4771 , piece_id=5F5TF72GWADPW4VY7T27HFLSZMBPTQRLNBFE2N26YUSQ7Y43KXVQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753138754 (565h57m45.105923852s) [noise=true , address=77.90.31.10:4520 , piece_id=A6IPQ4HSYUAEQFDJIF2524SFLMBEUCAMYJD5MNVMPOL4JHCXJPGQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753237640 (565h57m45.106022748s) [noise=true , address=145.79.227.13:4801 , piece_id=7D7G5GGMZXFOPQ5LSL73VK6MI3VJFPFJUYV5VCN2HBEIZTH4BE5Q, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753459406 (565h57m45.106244504s) [noise=true , address=145.79.227.2:4113 , piece_id=YFDZFNYX6XNZAIQJFCZNZO3WCUBL6UBQBVIO3J4SW5O4A2DKICZQ, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753541541 (565h57m45.106326639s) [noise=true , address=145.79.227.15:4873 , piece_id=I3FOTPN4FUGXFSNLOV6F55HE5SK6NACGQTVEIUPL2ZEG6I3QKK2A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.753773366 (565h57m45.106558463s) [noise=true , address=45.91.116.11:4117 , piece_id=2DTFQXZBN2A2PKBLP5VY7A7JR226OCXUEEN5WMSNSUVZ6EL3MI2A, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.754075913 (565h57m45.106861011s) [noise=true , address=145.79.227.4:4185 , piece_id=OV6KM67YXWDINQZVE6WKVXVP35P5XAXGDO4WKT2YWBQQINFDFDRA, seg_pos={0 0}, upload=498094]: Uploading piece... Sep 22 04:06:09.835461850 (565h57m45.188246959s) [upload=498094]: Next returned nil segment Sep 22 04:06:09.835473382 (565h57m45.18825849s) [upload=498094]: Waiting for error group managing segments... Sep 22 04:06:09.844018979 (565h57m45.196804077s) [noise=true , address=45.91.116.10:4082 , piece_id=CPA2ISXBRP6PF4JVYYGGY3BYF3DFVFN74NF75NIJTQ7CQIUXUXOQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.844981535 (565h57m45.197766633s) [noise=true , address=45.91.116.11:4117 , piece_id=2DTFQXZBN2A2PKBLP5VY7A7JR226OCXUEEN5WMSNSUVZ6EL3MI2A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845349495 (565h57m45.198134593s) [noise=true , address=45.91.116.10:4081 , piece_id=ZSXHNSUI7WP6MOI4CZKP64J3ZA2S2IOW2NCNL4IUYJRBF6K2PBCA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845438762 (565h57m45.19822387s) [noise=true , address=45.91.116.11:4118 , piece_id=3YG4ECFKLATKL4RL3NP4LFV5ZY7C6ERCYAHZDHTCIEFF7FKRS6PQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.845704431 (565h57m45.19848953s) [noise=true , address=45.91.116.13:4154 , piece_id=OZWZJGP3ILRFWQ4UZZBYZGWDLBGQVKCWGP6DSRJPNLYYXTM7HE7Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846133176 (565h57m45.198918274s) [noise=true , address=77.90.31.16:4733 , piece_id=HPGXY6LNBBWEITQKIRSS5FUWN27ANT53SXPBDB2OVQ2V7NOGWG5Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846306451 (565h57m45.199091559s) [noise=true , address=45.91.116.9:4262 , piece_id=SF76QXCSOSOFFOSQSCQPCHSE42RSGBSW7UA55CFSQV3IJ6SEVUSA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846642913 (565h57m45.199428021s) [noise=true , address=77.90.31.10:4520 , piece_id=A6IPQ4HSYUAEQFDJIF2524SFLMBEUCAMYJD5MNVMPOL4JHCXJPGQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:09.846727091 (565h57m45.199512189s) [noise=true , address=77.90.31.17:4771 , piece_id=5F5TF72GWADPW4VY7T27HFLSZMBPTQRLNBFE2N26YUSQ7Y43KXVQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: <nil> Sep 22 04:06:11.254886917 (565h57m46.607672015s) [noise=true , address=145.79.227.11:4730 , piece_id=MMMQHMZZTM2SDMZSVACKUY3PW2KIZ5QK4L4AF3LF32JGTNITLBXQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:11.254997294 (565h57m46.607782392s) [noise=true , address=145.79.227.11:4729 , piece_id=DPZMQNMF2KVN5GFDCFUTXGGFDS3RKUAFEKUIUP6JPOOXUE3465PQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: rpc: dial tcp 145.79.227.11:4729: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620098065 (565h58m14.972883173s) [noise=true , address=145.79.227.15:4873 , piece_id=I3FOTPN4FUGXFSNLOV6F55HE5SK6NACGQTVEIUPL2ZEG6I3QKK2A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620146045 (565h58m14.972931154s) [noise=true , address=145.79.227.4:4185 , piece_id=OV6KM67YXWDINQZVE6WKVXVP35P5XAXGDO4WKT2YWBQQINFDFDRA, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: rpc: dial tcp 145.79.227.4:4185: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620187432 (565h58m14.972972541s) [noise=true , address=145.79.227.13:4801 , piece_id=7D7G5GGMZXFOPQ5LSL73VK6MI3VJFPFJUYV5VCN2HBEIZTH4BE5Q, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: rpc: dial tcp 145.79.227.13:4801: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620225484 (565h58m14.973010583s) [noise=true , address=145.79.227.7:4295 , piece_id=LKOIQOEUD6SW3KMEKQ4357LKQO4YPOZOAR4Y7VCBCVFULEHB3B5A, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: rpc: dial tcp 145.79.227.7:4295: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:06:39.620269477 (565h58m14.973054585s) [noise=true , address=145.79.227.2:4113 , piece_id=YFDZFNYX6XNZAIQJFCZNZO3WCUBL6UBQBVIO3J4SW5O4A2DKICZQ, seg_pos={0 0}, upload=498094]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; context deadline exceeded Sep 22 04:06:39.620307199 (565h58m14.973092307s) [seg_pos={0 0}, upload=498094]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 22 04:06:39.637082920 (565h58m14.989868028s) [upload=498094]: Done uploading segments... Sep 22 04:06:39.637102205 (565h58m14.989887314s) [upload=498094]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.019232805s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [....X...X.XX...XXX...X.......X...X.........XX...............] duration: 459.064883ms FAIL (2025-09-22 04:06:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.084831ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X..X...X...X.X.....XX.....X........X...........X........] duration: 400.485202ms FAIL (2025-09-22 04:05:47 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:00:27.225037560 (565h52m2.577822668s) [upload=498009]: Starting upload Sep 22 04:00:27.225091682 (565h52m2.57787679s) [upload=498009]: Done starting upload Sep 22 04:00:27.225157835 (565h52m2.577942934s) [upload=498009]: Uploading segments... Sep 22 04:00:27.225402987 (565h52m2.578188095s) [upload=498009]: Got next segment. Inline: false Sep 22 04:00:27.225442009 (565h52m2.578227108s) [upload=498009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:00:27.225447650 (565h52m2.578232748s) [upload=498009]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:00:27.281446807 (565h52m2.634231915s) [seg_pos={0 0}, upload=498009]: Begin upload segment... Sep 22 04:00:27.281510987 (565h52m2.634296106s) [seg_pos={0 0}, upload=498009]: Done begin upload segment. Sep 22 04:00:27.281544140 (565h52m2.634329248s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281552215 (565h52m2.634337313s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281891010 (565h52m2.634676118s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281975549 (565h52m2.634760647s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282114299 (565h52m2.634899397s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282330105 (565h52m2.635115213s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282336927 (565h52m2.635122056s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282609028 (565h52m2.635394126s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282931613 (565h52m2.635716721s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283195969 (565h52m2.635981067s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283556596 (565h52m2.636341694s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283795284 (565h52m2.636580381s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284038360 (565h52m2.636823458s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284246300 (565h52m2.637031408s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284514453 (565h52m2.637299562s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.288615650 (565h52m2.641400758s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.363121154 (565h52m2.715906262s) [upload=498009]: Next returned nil segment Sep 22 04:00:27.363131884 (565h52m2.715916993s) [upload=498009]: Waiting for error group managing segments... Sep 22 04:00:27.370673817 (565h52m2.723458925s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.370896575 (565h52m2.723681673s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371119053 (565h52m2.723904161s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371314580 (565h52m2.724099688s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371762921 (565h52m2.724548019s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371819147 (565h52m2.724604255s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372170687 (565h52m2.724955795s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372196324 (565h52m2.724981432s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372552292 (565h52m2.7253374s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372641920 (565h52m2.725427028s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373567978 (565h52m2.726353086s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373750039 (565h52m2.726535138s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:43.507086986 (565h52m18.859872115s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:00:57.191201022 (565h52m32.54398613s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191216301 (565h52m32.544001409s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:00:57.191236569 (565h52m32.544021666s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191260884 (565h52m32.544045992s) [seg_pos={0 0}, upload=498009]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:00:57.224186591 (565h52m32.576971699s) [upload=498009]: Done uploading segments... Sep 22 04:00:57.224200608 (565h52m32.576985716s) [upload=498009]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 400.505901ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.....X...X.XX...XXX...X.......X...X.........XX..............] duration: 482.858504ms FAIL (2025-09-22 04:05:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 482.876196ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X..X...X...X.X.....XX.....X........X...........X.......] duration: 449.814834ms FAIL (2025-09-22 04:05:22 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:00:27.225037560 (565h52m2.577822668s) [upload=498009]: Starting upload Sep 22 04:00:27.225091682 (565h52m2.57787679s) [upload=498009]: Done starting upload Sep 22 04:00:27.225157835 (565h52m2.577942934s) [upload=498009]: Uploading segments... Sep 22 04:00:27.225402987 (565h52m2.578188095s) [upload=498009]: Got next segment. Inline: false Sep 22 04:00:27.225442009 (565h52m2.578227108s) [upload=498009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:00:27.225447650 (565h52m2.578232748s) [upload=498009]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:00:27.281446807 (565h52m2.634231915s) [seg_pos={0 0}, upload=498009]: Begin upload segment... Sep 22 04:00:27.281510987 (565h52m2.634296106s) [seg_pos={0 0}, upload=498009]: Done begin upload segment. Sep 22 04:00:27.281544140 (565h52m2.634329248s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281552215 (565h52m2.634337313s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281891010 (565h52m2.634676118s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281975549 (565h52m2.634760647s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282114299 (565h52m2.634899397s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282330105 (565h52m2.635115213s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282336927 (565h52m2.635122056s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282609028 (565h52m2.635394126s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282931613 (565h52m2.635716721s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283195969 (565h52m2.635981067s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283556596 (565h52m2.636341694s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283795284 (565h52m2.636580381s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284038360 (565h52m2.636823458s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284246300 (565h52m2.637031408s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284514453 (565h52m2.637299562s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.288615650 (565h52m2.641400758s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.363121154 (565h52m2.715906262s) [upload=498009]: Next returned nil segment Sep 22 04:00:27.363131884 (565h52m2.715916993s) [upload=498009]: Waiting for error group managing segments... Sep 22 04:00:27.370673817 (565h52m2.723458925s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.370896575 (565h52m2.723681673s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371119053 (565h52m2.723904161s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371314580 (565h52m2.724099688s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371762921 (565h52m2.724548019s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371819147 (565h52m2.724604255s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372170687 (565h52m2.724955795s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372196324 (565h52m2.724981432s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372552292 (565h52m2.7253374s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372641920 (565h52m2.725427028s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373567978 (565h52m2.726353086s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373750039 (565h52m2.726535138s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:43.507086986 (565h52m18.859872115s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:00:57.191201022 (565h52m32.54398613s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191216301 (565h52m32.544001409s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:00:57.191236569 (565h52m32.544021666s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191260884 (565h52m32.544045992s) [seg_pos={0 0}, upload=498009]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:00:57.224186591 (565h52m32.576971699s) [upload=498009]: Done uploading segments... Sep 22 04:00:57.224200608 (565h52m32.576985716s) [upload=498009]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 449.833209ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [......X...X.XX...XXX...X.......X...X.........XX.............] duration: 427.037748ms FAIL (2025-09-22 04:04:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 427.053167ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [......X..X...X...X.X.....XX.....X........X...........X......] duration: 845.233131ms FAIL (2025-09-22 04:04:29 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:00:27.225037560 (565h52m2.577822668s) [upload=498009]: Starting upload Sep 22 04:00:27.225091682 (565h52m2.57787679s) [upload=498009]: Done starting upload Sep 22 04:00:27.225157835 (565h52m2.577942934s) [upload=498009]: Uploading segments... Sep 22 04:00:27.225402987 (565h52m2.578188095s) [upload=498009]: Got next segment. Inline: false Sep 22 04:00:27.225442009 (565h52m2.578227108s) [upload=498009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:00:27.225447650 (565h52m2.578232748s) [upload=498009]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:00:27.281446807 (565h52m2.634231915s) [seg_pos={0 0}, upload=498009]: Begin upload segment... Sep 22 04:00:27.281510987 (565h52m2.634296106s) [seg_pos={0 0}, upload=498009]: Done begin upload segment. Sep 22 04:00:27.281544140 (565h52m2.634329248s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281552215 (565h52m2.634337313s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281891010 (565h52m2.634676118s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281975549 (565h52m2.634760647s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282114299 (565h52m2.634899397s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282330105 (565h52m2.635115213s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282336927 (565h52m2.635122056s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282609028 (565h52m2.635394126s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282931613 (565h52m2.635716721s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283195969 (565h52m2.635981067s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283556596 (565h52m2.636341694s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283795284 (565h52m2.636580381s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284038360 (565h52m2.636823458s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284246300 (565h52m2.637031408s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284514453 (565h52m2.637299562s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.288615650 (565h52m2.641400758s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.363121154 (565h52m2.715906262s) [upload=498009]: Next returned nil segment Sep 22 04:00:27.363131884 (565h52m2.715916993s) [upload=498009]: Waiting for error group managing segments... Sep 22 04:00:27.370673817 (565h52m2.723458925s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.370896575 (565h52m2.723681673s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371119053 (565h52m2.723904161s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371314580 (565h52m2.724099688s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371762921 (565h52m2.724548019s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371819147 (565h52m2.724604255s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372170687 (565h52m2.724955795s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372196324 (565h52m2.724981432s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372552292 (565h52m2.7253374s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372641920 (565h52m2.725427028s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373567978 (565h52m2.726353086s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373750039 (565h52m2.726535138s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:43.507086986 (565h52m18.859872115s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:00:57.191201022 (565h52m32.54398613s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191216301 (565h52m32.544001409s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:00:57.191236569 (565h52m32.544021666s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191260884 (565h52m32.544045992s) [seg_pos={0 0}, upload=498009]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:00:57.224186591 (565h52m32.576971699s) [upload=498009]: Done uploading segments... Sep 22 04:00:57.224200608 (565h52m32.576985716s) [upload=498009]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 845.256235ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.......X...X.XX...XXX...X.......X...X.........XX............] duration: 479.573996ms FAIL (2025-09-22 04:03:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 479.586219ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.......X..X...X...X.X.....XX.....X........X...........X.....] duration: 415.38989ms FAIL (2025-09-22 04:03:37 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:00:27.225037560 (565h52m2.577822668s) [upload=498009]: Starting upload Sep 22 04:00:27.225091682 (565h52m2.57787679s) [upload=498009]: Done starting upload Sep 22 04:00:27.225157835 (565h52m2.577942934s) [upload=498009]: Uploading segments... Sep 22 04:00:27.225402987 (565h52m2.578188095s) [upload=498009]: Got next segment. Inline: false Sep 22 04:00:27.225442009 (565h52m2.578227108s) [upload=498009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:00:27.225447650 (565h52m2.578232748s) [upload=498009]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:00:27.281446807 (565h52m2.634231915s) [seg_pos={0 0}, upload=498009]: Begin upload segment... Sep 22 04:00:27.281510987 (565h52m2.634296106s) [seg_pos={0 0}, upload=498009]: Done begin upload segment. Sep 22 04:00:27.281544140 (565h52m2.634329248s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281552215 (565h52m2.634337313s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281891010 (565h52m2.634676118s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281975549 (565h52m2.634760647s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282114299 (565h52m2.634899397s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282330105 (565h52m2.635115213s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282336927 (565h52m2.635122056s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282609028 (565h52m2.635394126s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282931613 (565h52m2.635716721s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283195969 (565h52m2.635981067s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283556596 (565h52m2.636341694s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283795284 (565h52m2.636580381s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284038360 (565h52m2.636823458s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284246300 (565h52m2.637031408s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284514453 (565h52m2.637299562s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.288615650 (565h52m2.641400758s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.363121154 (565h52m2.715906262s) [upload=498009]: Next returned nil segment Sep 22 04:00:27.363131884 (565h52m2.715916993s) [upload=498009]: Waiting for error group managing segments... Sep 22 04:00:27.370673817 (565h52m2.723458925s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.370896575 (565h52m2.723681673s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371119053 (565h52m2.723904161s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371314580 (565h52m2.724099688s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371762921 (565h52m2.724548019s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371819147 (565h52m2.724604255s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372170687 (565h52m2.724955795s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372196324 (565h52m2.724981432s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372552292 (565h52m2.7253374s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372641920 (565h52m2.725427028s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373567978 (565h52m2.726353086s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373750039 (565h52m2.726535138s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:43.507086986 (565h52m18.859872115s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:00:57.191201022 (565h52m32.54398613s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191216301 (565h52m32.544001409s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:00:57.191236569 (565h52m32.544021666s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191260884 (565h52m32.544045992s) [seg_pos={0 0}, upload=498009]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:00:57.224186591 (565h52m32.576971699s) [upload=498009]: Done uploading segments... Sep 22 04:00:57.224200608 (565h52m32.576985716s) [upload=498009]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 415.408135ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [........X...X.XX...XXX...X.......X...X.........XX...........] duration: 1.196498678s FAIL (2025-09-22 04:03:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.196522363s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........X..X...X...X.X.....XX.....X........X...........X....] duration: 808.364943ms FAIL (2025-09-22 04:03:14 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:00:27.225037560 (565h52m2.577822668s) [upload=498009]: Starting upload Sep 22 04:00:27.225091682 (565h52m2.57787679s) [upload=498009]: Done starting upload Sep 22 04:00:27.225157835 (565h52m2.577942934s) [upload=498009]: Uploading segments... Sep 22 04:00:27.225402987 (565h52m2.578188095s) [upload=498009]: Got next segment. Inline: false Sep 22 04:00:27.225442009 (565h52m2.578227108s) [upload=498009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:00:27.225447650 (565h52m2.578232748s) [upload=498009]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:00:27.281446807 (565h52m2.634231915s) [seg_pos={0 0}, upload=498009]: Begin upload segment... Sep 22 04:00:27.281510987 (565h52m2.634296106s) [seg_pos={0 0}, upload=498009]: Done begin upload segment. Sep 22 04:00:27.281544140 (565h52m2.634329248s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281552215 (565h52m2.634337313s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281891010 (565h52m2.634676118s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281975549 (565h52m2.634760647s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282114299 (565h52m2.634899397s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282330105 (565h52m2.635115213s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282336927 (565h52m2.635122056s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282609028 (565h52m2.635394126s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282931613 (565h52m2.635716721s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283195969 (565h52m2.635981067s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283556596 (565h52m2.636341694s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283795284 (565h52m2.636580381s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284038360 (565h52m2.636823458s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284246300 (565h52m2.637031408s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284514453 (565h52m2.637299562s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.288615650 (565h52m2.641400758s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.363121154 (565h52m2.715906262s) [upload=498009]: Next returned nil segment Sep 22 04:00:27.363131884 (565h52m2.715916993s) [upload=498009]: Waiting for error group managing segments... Sep 22 04:00:27.370673817 (565h52m2.723458925s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.370896575 (565h52m2.723681673s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371119053 (565h52m2.723904161s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371314580 (565h52m2.724099688s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371762921 (565h52m2.724548019s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371819147 (565h52m2.724604255s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372170687 (565h52m2.724955795s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372196324 (565h52m2.724981432s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372552292 (565h52m2.7253374s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372641920 (565h52m2.725427028s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373567978 (565h52m2.726353086s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373750039 (565h52m2.726535138s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:43.507086986 (565h52m18.859872115s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:00:57.191201022 (565h52m32.54398613s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191216301 (565h52m32.544001409s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:00:57.191236569 (565h52m32.544021666s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191260884 (565h52m32.544045992s) [seg_pos={0 0}, upload=498009]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:00:57.224186591 (565h52m32.576971699s) [upload=498009]: Done uploading segments... Sep 22 04:00:57.224200608 (565h52m32.576985716s) [upload=498009]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 808.391962ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.........X..X...X...X.X.....XX.....X........X...........X...] duration: 463.090886ms FAIL (2025-09-22 04:02:31 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:00:27.225037560 (565h52m2.577822668s) [upload=498009]: Starting upload Sep 22 04:00:27.225091682 (565h52m2.57787679s) [upload=498009]: Done starting upload Sep 22 04:00:27.225157835 (565h52m2.577942934s) [upload=498009]: Uploading segments... Sep 22 04:00:27.225402987 (565h52m2.578188095s) [upload=498009]: Got next segment. Inline: false Sep 22 04:00:27.225442009 (565h52m2.578227108s) [upload=498009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:00:27.225447650 (565h52m2.578232748s) [upload=498009]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:00:27.281446807 (565h52m2.634231915s) [seg_pos={0 0}, upload=498009]: Begin upload segment... Sep 22 04:00:27.281510987 (565h52m2.634296106s) [seg_pos={0 0}, upload=498009]: Done begin upload segment. Sep 22 04:00:27.281544140 (565h52m2.634329248s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281552215 (565h52m2.634337313s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281891010 (565h52m2.634676118s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281975549 (565h52m2.634760647s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282114299 (565h52m2.634899397s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282330105 (565h52m2.635115213s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282336927 (565h52m2.635122056s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282609028 (565h52m2.635394126s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282931613 (565h52m2.635716721s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283195969 (565h52m2.635981067s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283556596 (565h52m2.636341694s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283795284 (565h52m2.636580381s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284038360 (565h52m2.636823458s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284246300 (565h52m2.637031408s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284514453 (565h52m2.637299562s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.288615650 (565h52m2.641400758s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.363121154 (565h52m2.715906262s) [upload=498009]: Next returned nil segment Sep 22 04:00:27.363131884 (565h52m2.715916993s) [upload=498009]: Waiting for error group managing segments... Sep 22 04:00:27.370673817 (565h52m2.723458925s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.370896575 (565h52m2.723681673s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371119053 (565h52m2.723904161s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371314580 (565h52m2.724099688s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371762921 (565h52m2.724548019s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371819147 (565h52m2.724604255s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372170687 (565h52m2.724955795s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372196324 (565h52m2.724981432s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372552292 (565h52m2.7253374s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372641920 (565h52m2.725427028s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373567978 (565h52m2.726353086s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373750039 (565h52m2.726535138s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:43.507086986 (565h52m18.859872115s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:00:57.191201022 (565h52m32.54398613s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191216301 (565h52m32.544001409s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:00:57.191236569 (565h52m32.544021666s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191260884 (565h52m32.544045992s) [seg_pos={0 0}, upload=498009]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:00:57.224186591 (565h52m32.576971699s) [upload=498009]: Done uploading segments... Sep 22 04:00:57.224200608 (565h52m32.576985716s) [upload=498009]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 463.10985ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.........X...X.XX...XXX...X.......X...X.........XX..........] duration: 373.366119ms FAIL (2025-09-22 04:02:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 373.386117ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..........X..X...X...X.X.....XX.....X........X...........X..] duration: 447.470541ms FAIL (2025-09-22 04:01:28 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:00:27.225037560 (565h52m2.577822668s) [upload=498009]: Starting upload Sep 22 04:00:27.225091682 (565h52m2.57787679s) [upload=498009]: Done starting upload Sep 22 04:00:27.225157835 (565h52m2.577942934s) [upload=498009]: Uploading segments... Sep 22 04:00:27.225402987 (565h52m2.578188095s) [upload=498009]: Got next segment. Inline: false Sep 22 04:00:27.225442009 (565h52m2.578227108s) [upload=498009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:00:27.225447650 (565h52m2.578232748s) [upload=498009]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:00:27.281446807 (565h52m2.634231915s) [seg_pos={0 0}, upload=498009]: Begin upload segment... Sep 22 04:00:27.281510987 (565h52m2.634296106s) [seg_pos={0 0}, upload=498009]: Done begin upload segment. Sep 22 04:00:27.281544140 (565h52m2.634329248s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281552215 (565h52m2.634337313s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281891010 (565h52m2.634676118s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281975549 (565h52m2.634760647s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282114299 (565h52m2.634899397s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282330105 (565h52m2.635115213s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282336927 (565h52m2.635122056s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282609028 (565h52m2.635394126s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282931613 (565h52m2.635716721s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283195969 (565h52m2.635981067s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283556596 (565h52m2.636341694s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283795284 (565h52m2.636580381s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284038360 (565h52m2.636823458s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284246300 (565h52m2.637031408s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284514453 (565h52m2.637299562s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.288615650 (565h52m2.641400758s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.363121154 (565h52m2.715906262s) [upload=498009]: Next returned nil segment Sep 22 04:00:27.363131884 (565h52m2.715916993s) [upload=498009]: Waiting for error group managing segments... Sep 22 04:00:27.370673817 (565h52m2.723458925s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.370896575 (565h52m2.723681673s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371119053 (565h52m2.723904161s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371314580 (565h52m2.724099688s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371762921 (565h52m2.724548019s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371819147 (565h52m2.724604255s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372170687 (565h52m2.724955795s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372196324 (565h52m2.724981432s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372552292 (565h52m2.7253374s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372641920 (565h52m2.725427028s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373567978 (565h52m2.726353086s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373750039 (565h52m2.726535138s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:43.507086986 (565h52m18.859872115s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:00:57.191201022 (565h52m32.54398613s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191216301 (565h52m32.544001409s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:00:57.191236569 (565h52m32.544021666s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191260884 (565h52m32.544045992s) [seg_pos={0 0}, upload=498009]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:00:57.224186591 (565h52m32.576971699s) [upload=498009]: Done uploading segments... Sep 22 04:00:57.224200608 (565h52m32.576985716s) [upload=498009]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 447.489287ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........X...X.XX...XXX...X.......X...X.........XX.........] duration: 429.45881ms FAIL (2025-09-22 04:01:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 429.475472ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...........X..X...X...X.X.....XX.....X........X...........X.] duration: 477.045217ms FAIL (2025-09-22 04:00:57 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:00:27.225037560 (565h52m2.577822668s) [upload=498009]: Starting upload Sep 22 04:00:27.225091682 (565h52m2.57787679s) [upload=498009]: Done starting upload Sep 22 04:00:27.225157835 (565h52m2.577942934s) [upload=498009]: Uploading segments... Sep 22 04:00:27.225402987 (565h52m2.578188095s) [upload=498009]: Got next segment. Inline: false Sep 22 04:00:27.225442009 (565h52m2.578227108s) [upload=498009]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:00:27.225447650 (565h52m2.578232748s) [upload=498009]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:00:27.281446807 (565h52m2.634231915s) [seg_pos={0 0}, upload=498009]: Begin upload segment... Sep 22 04:00:27.281510987 (565h52m2.634296106s) [seg_pos={0 0}, upload=498009]: Done begin upload segment. Sep 22 04:00:27.281544140 (565h52m2.634329248s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281552215 (565h52m2.634337313s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281891010 (565h52m2.634676118s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.281975549 (565h52m2.634760647s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282114299 (565h52m2.634899397s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282330105 (565h52m2.635115213s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282336927 (565h52m2.635122056s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282609028 (565h52m2.635394126s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.282931613 (565h52m2.635716721s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283195969 (565h52m2.635981067s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283556596 (565h52m2.636341694s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.283795284 (565h52m2.636580381s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284038360 (565h52m2.636823458s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284246300 (565h52m2.637031408s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.284514453 (565h52m2.637299562s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.288615650 (565h52m2.641400758s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Uploading piece... Sep 22 04:00:27.363121154 (565h52m2.715906262s) [upload=498009]: Next returned nil segment Sep 22 04:00:27.363131884 (565h52m2.715916993s) [upload=498009]: Waiting for error group managing segments... Sep 22 04:00:27.370673817 (565h52m2.723458925s) [noise=true , address=45.91.116.10:4082 , piece_id=J67WLDSEQXTYCOQ2ZJRAS3XYARXOYBRVDRUT6YVJD7Y7SCMLT3MA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.370896575 (565h52m2.723681673s) [noise=true , address=45.91.116.6:4973 , piece_id=DYX7IOODLGZUJ7KRN7RAK4FPKLF2WFRWCSDTTPA76Z46337SOTWQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371119053 (565h52m2.723904161s) [noise=true , address=45.91.116.13:4154 , piece_id=Q3GTKMOLOCOIJ5JAZVCH7G4DWKLUATK2IPYWIT6UIYJYYA5PVK7Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371314580 (565h52m2.724099688s) [noise=true , address=45.91.116.4:4937 , piece_id=3VM6ZZM56HFZQUMCVKRBLBTIOZ3XIMIXPSUGE435EC5WGR4QIPVQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371762921 (565h52m2.724548019s) [noise=true , address=45.91.116.7:4010 , piece_id=UW6SR3QMPQEVJA3T2BBEBEPWCBCRGWGIPTSVJJ32FHKUPOIDUGEA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.371819147 (565h52m2.724604255s) [noise=true , address=77.90.31.11:4554 , piece_id=SZTRRC6DVYHL5HJPB6YZ4NH7A5NBP6QS3Y4JCV6Y5XYK62OKOBIQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372170687 (565h52m2.724955795s) [noise=true , address=45.91.116.5:4299 , piece_id=HJXNWXOEK3PYLCH6XKG5MXXHAWZXYRLVM4JH6XZLOYXUBWMSJYCQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372196324 (565h52m2.724981432s) [noise=true , address=45.91.116.9:4263 , piece_id=3O6XFC32VO3ZXMOD4YC7XFLYJMBJ45GY2X72MDLA7VWNFPVYPTNA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372552292 (565h52m2.7253374s) [noise=true , address=45.91.116.13:4153 , piece_id=Q4KIDKTDKWZGPBC7URJHODJYMWALPGV2O42FBEERX62LEAUIC2VQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.372641920 (565h52m2.725427028s) [noise=true , address=77.90.31.19:4843 , piece_id=EF2HOEIHAQMMRDYWT4KV3B2FCAN37CWFJEDWTRESJ4NZSAR3BICA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373567978 (565h52m2.726353086s) [noise=true , address=45.91.116.11:4118 , piece_id=QETB5C7HP3LSZL33FPHJXSNNKZIHZ63LGP3HPFXVGBHBKBI3SGMA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:27.373750039 (565h52m2.726535138s) [noise=true , address=77.90.31.17:4770 , piece_id=GTLNNGS6DHB5SVLUST4AA5CRC7BYK7S3L4QXFDDPBWUZOLUHTDRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: <nil> Sep 22 04:00:43.507086986 (565h52m18.859872115s) [noise=true , address=145.79.227.14:4838 , piece_id=QV6ZQYPCSIPRMER3UXA7UXLV7LHGVB3B3DFBL6CHFTEON7YNYQFQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:00:57.191201022 (565h52m32.54398613s) [noise=true , address=145.79.227.17:4969 , piece_id=ZUKGVHMPJTJ632KJPPS3JUKLHHON2QL2BLFG3UDRRG4MC2GKJSRA, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: rpc: dial tcp 145.79.227.17:4969: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191216301 (565h52m32.544001409s) [noise=true , address=145.79.227.11:4732 , piece_id=ZDHSZ75C6GIDALK2XUX6UUD6JEO44KUMUN6XNKZT5SWN665A4BPQ, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:00:57.191236569 (565h52m32.544021666s) [noise=true , address=145.79.227.4:4186 , piece_id=NGJRNH76HNF3CJXYILFN72LKAQ5FPXYAN6QBQDWJADP5C3ZEER5Q, seg_pos={0 0}, upload=498009]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:00:57.191260884 (565h52m32.544045992s) [seg_pos={0 0}, upload=498009]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:00:57.224186591 (565h52m32.576971699s) [upload=498009]: Done uploading segments... Sep 22 04:00:57.224200608 (565h52m32.576985716s) [upload=498009]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 477.062359ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X..........X...X.XX...XXX...X.......X...X.........XX........] duration: 488.86501ms FAIL (2025-09-22 04:00:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 488.880389ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X..........X...X.XX...XXX...X.......X...X.........XX.......] duration: 400.831741ms FAIL (2025-09-22 03:59:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 400.851957ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.............X..X...X...X.X.....XX.....X........X...........] duration: 513.475766ms FAIL (2025-09-22 03:59:32 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 513.503598ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..............X..X...X...X.X.....XX.....X........X..........] duration: 435.133211ms FAIL (2025-09-22 03:59:02 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 435.180261ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..X..........X...X.XX...XXX...X.......X...X.........XX......] duration: 407.066399ms FAIL (2025-09-22 03:58:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 407.081868ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...X..........X...X.XX...XXX...X.......X...X.........XX.....] duration: 744.649881ms FAIL (2025-09-22 03:58:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 744.668746ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...............X..X...X...X.X.....XX.....X........X.........] duration: 605.852378ms FAIL (2025-09-22 03:57:59 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 605.878777ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X...X..........X...X.XX...XXX...X.......X...X.........XX....] duration: 385.538853ms FAIL (2025-09-22 03:57:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 385.559091ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [................X..X...X...X.X.....XX.....X........X........] duration: 374.430727ms FAIL (2025-09-22 03:57:08 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 374.451465ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)