Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.X...X..........X...X.XX...XXX...X.......X...X.........XX...] duration: 381.211783ms FAIL (2025-09-22 03:56:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 381.248523ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................X..X...X...X.X.....XX.....X........X.......] duration: 384.595694ms FAIL (2025-09-22 03:56:32 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 384.613447ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..................X..X...X...X.X.....XX.....X........X......] duration: 485.299461ms FAIL (2025-09-22 03:55:49 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 485.319948ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X...X..........X...X.XX...XXX...X.......X...X.........XX..] duration: 424.620351ms FAIL (2025-09-22 03:55:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 424.643074ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [X..X...X..........X...X.XX...XXX...X.......X...X.........XX.] duration: 427.082282ms FAIL (2025-09-22 03:55:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 427.093242ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.X..X...X..........X...X.XX...XXX...X.......X...X.........XX] duration: 30.000780039s FAIL (2025-09-22 03:55:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000791942s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................X..X...X...X.X.....XX.....X........X.....] duration: 599.296377ms FAIL (2025-09-22 03:54:49 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 599.31955ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X..X...X..........X...X.XX...XXX...X.......X...X.........X] duration: 30.001468538s FAIL (2025-09-22 03:54:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001486051s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [....................X..X...X...X.X.....XX.....X........X....] duration: 368.354237ms FAIL (2025-09-22 03:54:06 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 368.379905ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.....................X..X...X...X.X.....XX.....X........X...] duration: 718.762581ms FAIL (2025-09-22 03:53:39 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 718.781848ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...X..X...X..........X...X.XX...XXX...X.......X...X.........] duration: 422.394866ms FAIL (2025-09-22 03:53:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 422.409944ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [......................X..X...X...X.X.....XX.....X........X..] duration: 414.661706ms FAIL (2025-09-22 03:53:10 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 414.693576ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.......................X..X...X...X.X.....XX.....X........X.] duration: 399.172507ms FAIL (2025-09-22 03:52:37 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:52:07.445901037 (565h43m42.798686145s) [upload=497884]: Starting upload Sep 22 03:52:07.445953956 (565h43m42.798739064s) [upload=497884]: Done starting upload Sep 22 03:52:07.446058061 (565h43m42.798843159s) [upload=497884]: Uploading segments... Sep 22 03:52:07.446179639 (565h43m42.798964747s) [upload=497884]: Got next segment. Inline: false Sep 22 03:52:07.446212291 (565h43m42.798997399s) [upload=497884]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:52:07.446216208 (565h43m42.799001306s) [upload=497884]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:52:07.477553364 (565h43m42.830338472s) [seg_pos={0 0}, upload=497884]: Begin upload segment... Sep 22 03:52:07.477596274 (565h43m42.830381382s) [seg_pos={0 0}, upload=497884]: Done begin upload segment. Sep 22 03:52:07.477616933 (565h43m42.830402042s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477627823 (565h43m42.830412922s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477924590 (565h43m42.830709699s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.477928808 (565h43m42.830713906s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478146296 (565h43m42.830931394s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478202522 (565h43m42.83098763s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478356060 (565h43m42.831141168s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478445358 (565h43m42.831230466s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478565743 (565h43m42.831350851s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478668306 (565h43m42.831453404s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.478916772 (565h43m42.83170188s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479154018 (565h43m42.831939116s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479366857 (565h43m42.832151966s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479580258 (565h43m42.832365355s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.479819165 (565h43m42.832604264s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.480057242 (565h43m42.83284235s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Uploading piece... Sep 22 03:52:07.569378260 (565h43m42.922163359s) [upload=497884]: Next returned nil segment Sep 22 03:52:07.569391064 (565h43m42.922176162s) [upload=497884]: Waiting for error group managing segments... Sep 22 03:52:07.577925991 (565h43m42.930711089s) [noise=true , address=45.91.116.14:4191 , piece_id=THA4YD4CWWVU4IHMQ5CEKL7SJYNX27LOI4WW6SKCWOT2NBNDHJVA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.578312737 (565h43m42.931097835s) [noise=true , address=45.91.116.9:4262 , piece_id=2BY3CFA62NJQHXM67ZK4VCH7BFQC5IJPKKCMTGHID2OJMTR2KPQQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.579470048 (565h43m42.932255147s) [noise=true , address=77.90.31.14:4663 , piece_id=D76DBTLS35OJERBZGDCGU25TZPB4DRBC4564LDXKEPAN4TSF5O6Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580139265 (565h43m42.932924363s) [noise=true , address=45.91.116.6:4973 , piece_id=AXHTCC6THFBWIKND7NS3ZT75U7VOQ4SQT7VUYPDPMJG6FEARG47A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580212783 (565h43m42.932997911s) [noise=true , address=45.91.116.7:4010 , piece_id=2XF5ZHECFARK6FEUXNITKMWBIZDNF3OXJAQ7UMYCVIBBM6LFTY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.580485314 (565h43m42.933270412s) [noise=true , address=45.91.116.12:4225 , piece_id=COL7JCV2FKI3VOQOMWEWBCYT3ACAMOB4AWRD5PXJ7T5FXWFC2OSA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581112260 (565h43m42.933897358s) [noise=true , address=77.90.31.17:4772 , piece_id=BZNIMS6SDLK6V4RD2ZXSFMRUDNUWUZ6JBRHCM4MAXU2Q44ICXCOA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581180158 (565h43m42.933965286s) [noise=true , address=45.91.116.5:4297 , piece_id=CJ4FEUON453MSFK6DU35CE76467LRBTT6VVD7O3ID75J4VUTB5FA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.581448662 (565h43m42.934233759s) [noise=true , address=45.91.116.10:4082 , piece_id=YOW3RZGXYLHAHSXY7U5AER2NKVZ2K5GNQINP5SOVGXXXDWWXLAAQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.582765022 (565h43m42.93555015s) [noise=true , address=45.91.116.8:4048 , piece_id=PXKOZNETGAAK2RHVSKZ2IZ2WRYXV5NOACKR3ZHCQO7E5SK7BDY3Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.583356442 (565h43m42.936141539s) [noise=true , address=77.90.31.13:4627 , piece_id=M5VFSALVJAPNTWIYLKS3OMRLGP3UL5QUZDW4WM7W64O26BZ2JE6A, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:07.584306975 (565h43m42.937092113s) [noise=true , address=77.90.31.15:4699 , piece_id=XZ3ITOT2SKGIIHRDMZSP5BQBGN3TMX2JUVFM76UW2KFJ2U4SRT7Q, seg_pos={0 0}, upload=497884]: Done uploading piece. err: <nil> Sep 22 03:52:37.409298429 (565h44m12.762083537s) [noise=true , address=145.79.227.13:4804 , piece_id=OXHTES2BZX3VXUN4CPSKW3S5RFGAW654MFZ7NP76TC7NGFOGTHGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: rpc: dial tcp 145.79.227.13:4804: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409345146 (565h44m12.762130244s) [noise=true , address=145.79.227.4:4187 , piece_id=NDZJL7FMVFIS5XLMCGH67Q3D6DA5G46WNXKAKLF2HAD2I3OUJ7XA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409394198 (565h44m12.762179306s) [noise=true , address=145.79.227.11:4732 , piece_id=REWAR2B4AXRWBURIBF674U4JMVGS4TORQKNLUT7OMN2ES55UTOCQ, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409417963 (565h44m12.762203061s) [noise=true , address=145.79.227.5:4222 , piece_id=Y2HTSKQT2KZBCGENLDBMG64WYXOG7ARGYTTBOAIHPHL5CLDF4WGA, seg_pos={0 0}, upload=497884]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:52:37.409436477 (565h44m12.762221585s) [seg_pos={0 0}, upload=497884]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:52:37.423645742 (565h44m12.77643085s) [upload=497884]: Done uploading segments... Sep 22 03:52:37.423660941 (565h44m12.776446039s) [upload=497884]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 399.189448ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [....X..X...X..........X...X.XX...XXX...X.......X...X........] duration: 781.579302ms FAIL (2025-09-22 03:52:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 781.598339ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.........................X..X...X...X.X.....XX.....X........] duration: 788.836507ms FAIL (2025-09-22 03:51:29 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 788.863068ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.....X..X...X..........X...X.XX...XXX...X.......X...X.......] duration: 393.070245ms FAIL (2025-09-22 03:50:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 393.083361ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..........................X..X...X...X.X.....XX.....X.......] duration: 430.652334ms FAIL (2025-09-22 03:50:53 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 430.67698ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X.....X..X...X..........X...X.XX...XXX...X.......X...X......] duration: 593.911672ms FAIL (2025-09-22 03:50:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 593.931258ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...........................X..X...X...X.X.....XX.....X......] duration: 454.785337ms FAIL (2025-09-22 03:50:10 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 454.812018ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [XX.....X..X...X..........X...X.XX...XXX...X.......X...X.....] duration: 406.032804ms FAIL (2025-09-22 03:49:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 406.05211ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.454048523s FAIL (2025-09-22 03:49:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.454067108s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.XX.....X..X...X..........X...X.XX...XXX...X.......X...X....] duration: 392.665957ms FAIL (2025-09-22 03:49:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 392.684292ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [............................X..X...X...X.X.....XX.....X.....] duration: 415.283481ms FAIL (2025-09-22 03:48:56 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 415.302507ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 2.328236569s FAIL (2025-09-22 03:48:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.328261676s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.............................X..X...X...X.X.....XX.....X....] duration: 438.955632ms FAIL (2025-09-22 03:48:17 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 438.990857ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 2.185970157s FAIL (2025-09-22 03:48:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.185992911s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [..XX.....X..X...X..........X...X.XX...XXX...X.......X...X...] duration: 488.660999ms FAIL (2025-09-22 03:48:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 488.717383ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..............................X..X...X...X.X.....XX.....X...] duration: 403.080782ms FAIL (2025-09-22 03:47:48 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 403.109285ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [...XX.....X..X...X..........X...X.XX...XXX...X.......X...X..] duration: 409.279677ms FAIL (2025-09-22 03:47:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 409.301508ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [....XX.....X..X...X..........X...X.XX...XXX...X.......X...X.] duration: 404.162618ms FAIL (2025-09-22 03:47:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 404.176444ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....XX.....X..X...X..........X...X.XX...XXX...X.......X...X] duration: 30.001047905s FAIL (2025-09-22 03:47:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00106663s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...............................X..X...X...X.X.....XX.....X..] duration: 503.799355ms FAIL (2025-09-22 03:47:07 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 503.827208ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 2.286239942s FAIL (2025-09-22 03:46:59 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.286271151s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [................................X..X...X...X.X.....XX.....X.] duration: 612.161195ms FAIL (2025-09-22 03:46:25 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 612.176894ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......XX.....X..X...X..........X...X.XX...XXX...X.......X...] duration: 450.886872ms FAIL (2025-09-22 03:46:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 450.903774ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 2.225704601s FAIL (2025-09-22 03:46:03 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.225725521s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......XX.....X..X...X..........X...X.XX...XXX...X.......X..] duration: 399.193185ms FAIL (2025-09-22 03:45:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 399.406766ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.................................X..X...X...X.X.....XX.....X] duration: 30.021685245s FAIL (2025-09-22 03:45:44 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:45:14.747159999 (565h36m50.099945137s) [upload=497780]: Starting upload Sep 22 03:45:14.747238176 (565h36m50.100023304s) [upload=497780]: Done starting upload Sep 22 03:45:14.747303748 (565h36m50.100088887s) [upload=497780]: Uploading segments... Sep 22 03:45:14.747706776 (565h36m50.100491904s) [upload=497780]: Got next segment. Inline: false Sep 22 03:45:14.747755707 (565h36m50.100540836s) [upload=497780]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:45:14.747761568 (565h36m50.100546697s) [upload=497780]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:45:14.771470018 (565h36m50.124255147s) [seg_pos={0 0}, upload=497780]: Begin upload segment... Sep 22 03:45:14.771520885 (565h36m50.124306022s) [seg_pos={0 0}, upload=497780]: Done begin upload segment. Sep 22 03:45:14.771552664 (565h36m50.124337792s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.771561370 (565h36m50.124346498s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772081988 (565h36m50.124867115s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772112735 (565h36m50.124897864s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772494661 (565h36m50.1252798s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772633292 (565h36m50.12541842s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.772999660 (565h36m50.125784788s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773159429 (565h36m50.125944567s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773437190 (565h36m50.126222318s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773533090 (565h36m50.126318218s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773873380 (565h36m50.126658507s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.773912132 (565h36m50.12669726s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774259313 (565h36m50.127044452s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774278519 (565h36m50.127063648s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774604381 (565h36m50.12738951s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.774974786 (565h36m50.127759914s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Uploading piece... Sep 22 03:45:14.892566525 (565h36m50.245351633s) [upload=497780]: Next returned nil segment Sep 22 03:45:14.892580211 (565h36m50.245365319s) [upload=497780]: Waiting for error group managing segments... Sep 22 03:45:14.901916712 (565h36m50.25470182s) [noise=true , address=45.91.116.8:4045 , piece_id=YYOAOI5ZJJU2IZGVZJRJ7MAK5CMSA5ILHGG54VO2NUPBCHDCMTPQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902525644 (565h36m50.255310752s) [noise=true , address=45.91.116.14:4189 , piece_id=LLAJ7XCR4YI5EMZ6PZUIGXEOR3RMTELVHDB76Q2W6EPFRDLFYIUQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.902985557 (565h36m50.255770675s) [noise=true , address=45.91.116.8:4047 , piece_id=UBJK4HXMSPB4OMYPGSB7GRDQOLBT7KLMKGQV2BZU5ZBTU3LVGDKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903310025 (565h36m50.256095154s) [noise=true , address=45.91.116.14:4190 , piece_id=33A6GFDAKSKGHKRDDL2RWPDO263ZKMNGALOHTRSV4BZIHR3UR3OA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.903452994 (565h36m50.256238102s) [noise=true , address=45.91.116.10:4083 , piece_id=NCB3MNPDEOJCM5DKQMKS5OZA73OKFTZ34W3DQV23JRUASN3T5EUA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904038263 (565h36m50.256823391s) [noise=true , address=45.91.116.11:4119 , piece_id=NLBNS35XXU7J7JMIRABNUJOVTJ27UY5RY2D76TA6AIX3THFYVBQA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904499398 (565h36m50.257284506s) [noise=true , address=77.90.31.19:4843 , piece_id=QODTZDEKUM2EY4LWWQCBP7OPQNX3NW6NEXUWKCM5BIXNEVBWJ5LQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.904711646 (565h36m50.257496764s) [noise=true , address=77.90.31.14:4662 , piece_id=IGEEK23TC45MDMUQQUQK3GE6LHB73DQMD4O754V42WAGN3KIFGVQ, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905216584 (565h36m50.258001702s) [noise=true , address=77.90.31.14:4661 , piece_id=GENOSX3OC7EUOJVVKD3UUNTIKEZQAI5LNZ4DJBEMAKFQRODIIO4A, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905229238 (565h36m50.258014365s) [noise=true , address=77.90.31.16:4735 , piece_id=J2WQCOSEDL74AGFJM2Y672PTYYOCFLJLSFSEQAPTRIYW4QQTF4CA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905787776 (565h36m50.258572904s) [noise=true , address=77.90.31.21:4877 , piece_id=CWBSNKKH2AA7LNONDMJDA34WRLELYZW5JD4MUMK34FTZHPGACCNA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:14.905895137 (565h36m50.258680245s) [noise=true , address=77.90.31.17:4770 , piece_id=CGEGRY33B4K7QQYH77OWW3KCDBM5DASUL7S25HDF5J5UJ7FKSLIA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: <nil> Sep 22 03:45:31.204265371 (565h37m6.55705051s) [noise=true , address=145.79.227.16:4909 , piece_id=EBCDPN6E6AJOHVADLXSBJXD5EJLDJGEVM7MKIQGXN5F35CDKKAAA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: rpc: dial tcp 145.79.227.16:4909: connect: no route to host; conn closed: op Sep 22 03:45:44.733652533 (565h37m20.086437641s) [noise=true , address=145.79.227.11:4730 , piece_id=5DP7JZLLVMVSK52OBTQCM7GEO7TMDWBYYCKOWRTOIKXPLRHLACMA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: i/o timeout; conn closed: op Sep 22 03:45:44.733668112 (565h37m20.08645322s) [noise=true , address=145.79.227.17:4969 , piece_id=DLLFH7V5GFJKNFBPG6OMONLINRZ75HMRA6LD777T3J2R6RF24MBA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733780202 (565h37m20.0865653s) [noise=true , address=145.79.227.6:4258 , piece_id=VYCY4JXKWTZIALRQKXYGO4ZRKYH2KVNNET6RXDSD2YO7XA3C2CKA, seg_pos={0 0}, upload=497780]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:45:44.733802023 (565h37m20.086587131s) [seg_pos={0 0}, upload=497780]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:45:44.753762991 (565h37m20.106548089s) [upload=497780]: Done uploading segments... Sep 22 03:45:44.753775224 (565h37m20.106560332s) [upload=497780]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.021702778s core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 2.120256062s FAIL (2025-09-22 03:45:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.120287371s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 2.128643172s FAIL (2025-09-22 03:45:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.12867974s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [........XX.....X..X...X..........X...X.XX...XXX...X.......X.] duration: 421.755743ms FAIL (2025-09-22 03:44:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 421.773748ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..................................X..X...X...X.X.....XX.....] duration: 426.753638ms FAIL (2025-09-22 03:44:37 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:41:53.821360598 (565h33m29.174145726s) [upload=497735]: Starting upload Sep 22 03:41:53.821432342 (565h33m29.174217471s) [upload=497735]: Done starting upload Sep 22 03:41:53.821573697 (565h33m29.174358826s) [upload=497735]: Uploading segments... Sep 22 03:41:53.821895693 (565h33m29.174680821s) [upload=497735]: Got next segment. Inline: false Sep 22 03:41:53.821989369 (565h33m29.174774496s) [upload=497735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:41:53.821995941 (565h33m29.174781069s) [upload=497735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:41:53.843161018 (565h33m29.195946156s) [seg_pos={0 0}, upload=497735]: Begin upload segment... Sep 22 03:41:53.843219267 (565h33m29.196004405s) [seg_pos={0 0}, upload=497735]: Done begin upload segment. Sep 22 03:41:53.843260845 (565h33m29.196045973s) [noise=true , address=45.91.116.8:4047 , piece_id=GWNXFH4LDIAYWDKNKVONYRFVTTKBQHLR72ZHHFZ7M2YDXCBXLRIQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.843270974 (565h33m29.196056102s) [noise=true , address=45.91.116.7:4009 , piece_id=3K2HO6HF3KR7PIJRDABE7FW4A2FVEKHNR3ANPMU7MWVILDCKYX6Q, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.843790710 (565h33m29.196575838s) [noise=true , address=145.79.227.6:4259 , piece_id=WORJPEIFUREEUUCOV3P4RPFEQLXGHSUD4ZBLNLN2OVJDUAHVTDHA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844019850 (565h33m29.196804988s) [noise=true , address=45.91.116.13:4154 , piece_id=M2Q6RNA3AKZUBYFE6OMIZOFTRX7ASU7D3SPCVUGUDLMIQ7TZIZWA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844360168 (565h33m29.197145296s) [noise=true , address=145.79.227.12:4765 , piece_id=UJQMQKN3YUVQ2AQNXBPYV2ZUUKWY7Z4F6FHG4DXKI6ARMA2TFIXQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844472439 (565h33m29.197257576s) [noise=true , address=77.90.31.21:4878 , piece_id=TPRVB7U4ZNR7BOYT4BC22JIDHV3IHBTWM76R25UK5JRYR4XUSI7A, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844820201 (565h33m29.19760533s) [noise=true , address=77.90.31.13:4628 , piece_id=BLD55WPW4WBWJ6GWBBJDXWIYRQNPCUBA3N5UFZDT4WFKMDJOKP3Q, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844922043 (565h33m29.197707171s) [noise=true , address=45.91.116.9:4261 , piece_id=KXFVZ2ES3MGT26OJRKEHYW5DPGPMEFUF2WTEZXEQFXZBLHEDILKA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.845271628 (565h33m29.198056767s) [noise=true , address=145.79.227.18:4005 , piece_id=27PBR6OCJYCV76LXCWGZ4RMHYHYELU7UUQWK77ONLAKO7OZP5RJA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.845900639 (565h33m29.198685767s) [noise=true , address=45.91.116.4:4939 , piece_id=X6PJQNIPP5DGU4LGUJZEXJXR7FAARPN77W5D6MBNRMHPDEPY25AA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.846045771 (565h33m29.1988309s) [noise=true , address=45.91.116.7:4010 , piece_id=CKBCDXJE3ATNSEDNH4DNFH3WL4GZMVMEXVXRLI7GRY3XFUU2WOMA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.846725577 (565h33m29.199510705s) [noise=true , address=77.90.31.19:4842 , piece_id=WOMDAGSW44HXQV7E6JQEQBZKHIYGLW25QPBRCPK2XCRT62MUAHYQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.847449616 (565h33m29.200234744s) [noise=true , address=77.90.31.16:4733 , piece_id=MHKBKDVWJ2RZQXRLMTO33LSX7YJWJ3PMZJSVRO3RXKDXP2F7CSVA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.847961646 (565h33m29.200746775s) [noise=true , address=45.91.116.11:4118 , piece_id=R6HKZQ2B5AQ4C7YHHPBON4SVHPIV2IJN7DMSY6H7BSX7WHTSPKAQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.849104080 (565h33m29.201889208s) [noise=true , address=77.90.31.12:4592 , piece_id=CKT6LHANOHVC4NHMRQ2SEEXFPQA7MHM44RKOISA45M26QAWIPDVA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.850005882 (565h33m29.202791011s) [noise=true , address=145.79.227.4:4186 , piece_id=DJEVDGISBPJJ4R37Z5RXWHVLOXZRPMMVC3PWO66H24JQHY46QAUA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.982635031 (565h33m29.335420168s) [upload=497735]: Next returned nil segment Sep 22 03:41:53.982652824 (565h33m29.335437952s) [upload=497735]: Waiting for error group managing segments... Sep 22 03:41:53.990044095 (565h33m29.342829232s) [noise=true , address=45.91.116.11:4118 , piece_id=R6HKZQ2B5AQ4C7YHHPBON4SVHPIV2IJN7DMSY6H7BSX7WHTSPKAQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.990531790 (565h33m29.343316918s) [noise=true , address=45.91.116.13:4154 , piece_id=M2Q6RNA3AKZUBYFE6OMIZOFTRX7ASU7D3SPCVUGUDLMIQ7TZIZWA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991047046 (565h33m29.343832184s) [noise=true , address=45.91.116.9:4261 , piece_id=KXFVZ2ES3MGT26OJRKEHYW5DPGPMEFUF2WTEZXEQFXZBLHEDILKA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991466263 (565h33m29.344251392s) [noise=true , address=45.91.116.7:4009 , piece_id=3K2HO6HF3KR7PIJRDABE7FW4A2FVEKHNR3ANPMU7MWVILDCKYX6Q, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991769693 (565h33m29.344554831s) [noise=true , address=45.91.116.4:4939 , piece_id=X6PJQNIPP5DGU4LGUJZEXJXR7FAARPN77W5D6MBNRMHPDEPY25AA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.992458005 (565h33m29.345243133s) [noise=true , address=77.90.31.19:4842 , piece_id=WOMDAGSW44HXQV7E6JQEQBZKHIYGLW25QPBRCPK2XCRT62MUAHYQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.992505003 (565h33m29.345290131s) [noise=true , address=45.91.116.7:4010 , piece_id=CKBCDXJE3ATNSEDNH4DNFH3WL4GZMVMEXVXRLI7GRY3XFUU2WOMA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.993159922 (565h33m29.34594505s) [noise=true , address=77.90.31.12:4592 , piece_id=CKT6LHANOHVC4NHMRQ2SEEXFPQA7MHM44RKOISA45M26QAWIPDVA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.993274718 (565h33m29.346059845s) [noise=true , address=45.91.116.8:4047 , piece_id=GWNXFH4LDIAYWDKNKVONYRFVTTKBQHLR72ZHHFZ7M2YDXCBXLRIQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.994692838 (565h33m29.347477966s) [noise=true , address=77.90.31.21:4878 , piece_id=TPRVB7U4ZNR7BOYT4BC22JIDHV3IHBTWM76R25UK5JRYR4XUSI7A, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.994841236 (565h33m29.347626365s) [noise=true , address=77.90.31.13:4628 , piece_id=BLD55WPW4WBWJ6GWBBJDXWIYRQNPCUBA3N5UFZDT4WFKMDJOKP3Q, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.995361182 (565h33m29.348146311s) [noise=true , address=77.90.31.16:4733 , piece_id=MHKBKDVWJ2RZQXRLMTO33LSX7YJWJ3PMZJSVRO3RXKDXP2F7CSVA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:42:23.809279587 (565h33m59.162064714s) [noise=true , address=145.79.227.4:4186 , piece_id=DJEVDGISBPJJ4R37Z5RXWHVLOXZRPMMVC3PWO66H24JQHY46QAUA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:42:23.809421593 (565h33m59.162206721s) [noise=true , address=145.79.227.12:4765 , piece_id=UJQMQKN3YUVQ2AQNXBPYV2ZUUKWY7Z4F6FHG4DXKI6ARMA2TFIXQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:42:23.809560264 (565h33m59.162345402s) [noise=true , address=145.79.227.6:4259 , piece_id=WORJPEIFUREEUUCOV3P4RPFEQLXGHSUD4ZBLNLN2OVJDUAHVTDHA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:42:23.809598205 (565h33m59.162383333s) [noise=true , address=145.79.227.18:4005 , piece_id=27PBR6OCJYCV76LXCWGZ4RMHYHYELU7UUQWK77ONLAKO7OZP5RJA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; context deadline exceeded Sep 22 03:42:23.809711137 (565h33m59.162496265s) [seg_pos={0 0}, upload=497735]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:42:23.825581299 (565h33m59.178366426s) [upload=497735]: Done uploading segments... Sep 22 03:42:23.825600274 (565h33m59.178385412s) [upload=497735]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 426.78075ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 4.926941378s FAIL (2025-09-22 03:44:31 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.926961085s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 2.426183507s FAIL (2025-09-22 03:44:28 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.426222601s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...................................X..X...X...X.X.....XX....] duration: 496.225633ms FAIL (2025-09-22 03:44:12 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:41:53.821360598 (565h33m29.174145726s) [upload=497735]: Starting upload Sep 22 03:41:53.821432342 (565h33m29.174217471s) [upload=497735]: Done starting upload Sep 22 03:41:53.821573697 (565h33m29.174358826s) [upload=497735]: Uploading segments... Sep 22 03:41:53.821895693 (565h33m29.174680821s) [upload=497735]: Got next segment. Inline: false Sep 22 03:41:53.821989369 (565h33m29.174774496s) [upload=497735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:41:53.821995941 (565h33m29.174781069s) [upload=497735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:41:53.843161018 (565h33m29.195946156s) [seg_pos={0 0}, upload=497735]: Begin upload segment... Sep 22 03:41:53.843219267 (565h33m29.196004405s) [seg_pos={0 0}, upload=497735]: Done begin upload segment. Sep 22 03:41:53.843260845 (565h33m29.196045973s) [noise=true , address=45.91.116.8:4047 , piece_id=GWNXFH4LDIAYWDKNKVONYRFVTTKBQHLR72ZHHFZ7M2YDXCBXLRIQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.843270974 (565h33m29.196056102s) [noise=true , address=45.91.116.7:4009 , piece_id=3K2HO6HF3KR7PIJRDABE7FW4A2FVEKHNR3ANPMU7MWVILDCKYX6Q, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.843790710 (565h33m29.196575838s) [noise=true , address=145.79.227.6:4259 , piece_id=WORJPEIFUREEUUCOV3P4RPFEQLXGHSUD4ZBLNLN2OVJDUAHVTDHA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844019850 (565h33m29.196804988s) [noise=true , address=45.91.116.13:4154 , piece_id=M2Q6RNA3AKZUBYFE6OMIZOFTRX7ASU7D3SPCVUGUDLMIQ7TZIZWA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844360168 (565h33m29.197145296s) [noise=true , address=145.79.227.12:4765 , piece_id=UJQMQKN3YUVQ2AQNXBPYV2ZUUKWY7Z4F6FHG4DXKI6ARMA2TFIXQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844472439 (565h33m29.197257576s) [noise=true , address=77.90.31.21:4878 , piece_id=TPRVB7U4ZNR7BOYT4BC22JIDHV3IHBTWM76R25UK5JRYR4XUSI7A, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844820201 (565h33m29.19760533s) [noise=true , address=77.90.31.13:4628 , piece_id=BLD55WPW4WBWJ6GWBBJDXWIYRQNPCUBA3N5UFZDT4WFKMDJOKP3Q, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844922043 (565h33m29.197707171s) [noise=true , address=45.91.116.9:4261 , piece_id=KXFVZ2ES3MGT26OJRKEHYW5DPGPMEFUF2WTEZXEQFXZBLHEDILKA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.845271628 (565h33m29.198056767s) [noise=true , address=145.79.227.18:4005 , piece_id=27PBR6OCJYCV76LXCWGZ4RMHYHYELU7UUQWK77ONLAKO7OZP5RJA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.845900639 (565h33m29.198685767s) [noise=true , address=45.91.116.4:4939 , piece_id=X6PJQNIPP5DGU4LGUJZEXJXR7FAARPN77W5D6MBNRMHPDEPY25AA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.846045771 (565h33m29.1988309s) [noise=true , address=45.91.116.7:4010 , piece_id=CKBCDXJE3ATNSEDNH4DNFH3WL4GZMVMEXVXRLI7GRY3XFUU2WOMA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.846725577 (565h33m29.199510705s) [noise=true , address=77.90.31.19:4842 , piece_id=WOMDAGSW44HXQV7E6JQEQBZKHIYGLW25QPBRCPK2XCRT62MUAHYQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.847449616 (565h33m29.200234744s) [noise=true , address=77.90.31.16:4733 , piece_id=MHKBKDVWJ2RZQXRLMTO33LSX7YJWJ3PMZJSVRO3RXKDXP2F7CSVA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.847961646 (565h33m29.200746775s) [noise=true , address=45.91.116.11:4118 , piece_id=R6HKZQ2B5AQ4C7YHHPBON4SVHPIV2IJN7DMSY6H7BSX7WHTSPKAQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.849104080 (565h33m29.201889208s) [noise=true , address=77.90.31.12:4592 , piece_id=CKT6LHANOHVC4NHMRQ2SEEXFPQA7MHM44RKOISA45M26QAWIPDVA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.850005882 (565h33m29.202791011s) [noise=true , address=145.79.227.4:4186 , piece_id=DJEVDGISBPJJ4R37Z5RXWHVLOXZRPMMVC3PWO66H24JQHY46QAUA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.982635031 (565h33m29.335420168s) [upload=497735]: Next returned nil segment Sep 22 03:41:53.982652824 (565h33m29.335437952s) [upload=497735]: Waiting for error group managing segments... Sep 22 03:41:53.990044095 (565h33m29.342829232s) [noise=true , address=45.91.116.11:4118 , piece_id=R6HKZQ2B5AQ4C7YHHPBON4SVHPIV2IJN7DMSY6H7BSX7WHTSPKAQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.990531790 (565h33m29.343316918s) [noise=true , address=45.91.116.13:4154 , piece_id=M2Q6RNA3AKZUBYFE6OMIZOFTRX7ASU7D3SPCVUGUDLMIQ7TZIZWA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991047046 (565h33m29.343832184s) [noise=true , address=45.91.116.9:4261 , piece_id=KXFVZ2ES3MGT26OJRKEHYW5DPGPMEFUF2WTEZXEQFXZBLHEDILKA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991466263 (565h33m29.344251392s) [noise=true , address=45.91.116.7:4009 , piece_id=3K2HO6HF3KR7PIJRDABE7FW4A2FVEKHNR3ANPMU7MWVILDCKYX6Q, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991769693 (565h33m29.344554831s) [noise=true , address=45.91.116.4:4939 , piece_id=X6PJQNIPP5DGU4LGUJZEXJXR7FAARPN77W5D6MBNRMHPDEPY25AA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.992458005 (565h33m29.345243133s) [noise=true , address=77.90.31.19:4842 , piece_id=WOMDAGSW44HXQV7E6JQEQBZKHIYGLW25QPBRCPK2XCRT62MUAHYQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.992505003 (565h33m29.345290131s) [noise=true , address=45.91.116.7:4010 , piece_id=CKBCDXJE3ATNSEDNH4DNFH3WL4GZMVMEXVXRLI7GRY3XFUU2WOMA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.993159922 (565h33m29.34594505s) [noise=true , address=77.90.31.12:4592 , piece_id=CKT6LHANOHVC4NHMRQ2SEEXFPQA7MHM44RKOISA45M26QAWIPDVA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.993274718 (565h33m29.346059845s) [noise=true , address=45.91.116.8:4047 , piece_id=GWNXFH4LDIAYWDKNKVONYRFVTTKBQHLR72ZHHFZ7M2YDXCBXLRIQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.994692838 (565h33m29.347477966s) [noise=true , address=77.90.31.21:4878 , piece_id=TPRVB7U4ZNR7BOYT4BC22JIDHV3IHBTWM76R25UK5JRYR4XUSI7A, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.994841236 (565h33m29.347626365s) [noise=true , address=77.90.31.13:4628 , piece_id=BLD55WPW4WBWJ6GWBBJDXWIYRQNPCUBA3N5UFZDT4WFKMDJOKP3Q, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.995361182 (565h33m29.348146311s) [noise=true , address=77.90.31.16:4733 , piece_id=MHKBKDVWJ2RZQXRLMTO33LSX7YJWJ3PMZJSVRO3RXKDXP2F7CSVA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:42:23.809279587 (565h33m59.162064714s) [noise=true , address=145.79.227.4:4186 , piece_id=DJEVDGISBPJJ4R37Z5RXWHVLOXZRPMMVC3PWO66H24JQHY46QAUA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:42:23.809421593 (565h33m59.162206721s) [noise=true , address=145.79.227.12:4765 , piece_id=UJQMQKN3YUVQ2AQNXBPYV2ZUUKWY7Z4F6FHG4DXKI6ARMA2TFIXQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:42:23.809560264 (565h33m59.162345402s) [noise=true , address=145.79.227.6:4259 , piece_id=WORJPEIFUREEUUCOV3P4RPFEQLXGHSUD4ZBLNLN2OVJDUAHVTDHA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:42:23.809598205 (565h33m59.162383333s) [noise=true , address=145.79.227.18:4005 , piece_id=27PBR6OCJYCV76LXCWGZ4RMHYHYELU7UUQWK77ONLAKO7OZP5RJA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; context deadline exceeded Sep 22 03:42:23.809711137 (565h33m59.162496265s) [seg_pos={0 0}, upload=497735]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:42:23.825581299 (565h33m59.178366426s) [upload=497735]: Done uploading segments... Sep 22 03:42:23.825600274 (565h33m59.178385412s) [upload=497735]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 496.255279ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.........XX.....X..X...X..........X...X.XX...XXX...X.......X] duration: 30.000755072s FAIL (2025-09-22 03:44:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000767375s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [....................................X..X...X...X.X.....XX...] duration: 447.612737ms FAIL (2025-09-22 03:43:52 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:41:53.821360598 (565h33m29.174145726s) [upload=497735]: Starting upload Sep 22 03:41:53.821432342 (565h33m29.174217471s) [upload=497735]: Done starting upload Sep 22 03:41:53.821573697 (565h33m29.174358826s) [upload=497735]: Uploading segments... Sep 22 03:41:53.821895693 (565h33m29.174680821s) [upload=497735]: Got next segment. Inline: false Sep 22 03:41:53.821989369 (565h33m29.174774496s) [upload=497735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:41:53.821995941 (565h33m29.174781069s) [upload=497735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:41:53.843161018 (565h33m29.195946156s) [seg_pos={0 0}, upload=497735]: Begin upload segment... Sep 22 03:41:53.843219267 (565h33m29.196004405s) [seg_pos={0 0}, upload=497735]: Done begin upload segment. Sep 22 03:41:53.843260845 (565h33m29.196045973s) [noise=true , address=45.91.116.8:4047 , piece_id=GWNXFH4LDIAYWDKNKVONYRFVTTKBQHLR72ZHHFZ7M2YDXCBXLRIQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.843270974 (565h33m29.196056102s) [noise=true , address=45.91.116.7:4009 , piece_id=3K2HO6HF3KR7PIJRDABE7FW4A2FVEKHNR3ANPMU7MWVILDCKYX6Q, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.843790710 (565h33m29.196575838s) [noise=true , address=145.79.227.6:4259 , piece_id=WORJPEIFUREEUUCOV3P4RPFEQLXGHSUD4ZBLNLN2OVJDUAHVTDHA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844019850 (565h33m29.196804988s) [noise=true , address=45.91.116.13:4154 , piece_id=M2Q6RNA3AKZUBYFE6OMIZOFTRX7ASU7D3SPCVUGUDLMIQ7TZIZWA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844360168 (565h33m29.197145296s) [noise=true , address=145.79.227.12:4765 , piece_id=UJQMQKN3YUVQ2AQNXBPYV2ZUUKWY7Z4F6FHG4DXKI6ARMA2TFIXQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844472439 (565h33m29.197257576s) [noise=true , address=77.90.31.21:4878 , piece_id=TPRVB7U4ZNR7BOYT4BC22JIDHV3IHBTWM76R25UK5JRYR4XUSI7A, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844820201 (565h33m29.19760533s) [noise=true , address=77.90.31.13:4628 , piece_id=BLD55WPW4WBWJ6GWBBJDXWIYRQNPCUBA3N5UFZDT4WFKMDJOKP3Q, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844922043 (565h33m29.197707171s) [noise=true , address=45.91.116.9:4261 , piece_id=KXFVZ2ES3MGT26OJRKEHYW5DPGPMEFUF2WTEZXEQFXZBLHEDILKA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.845271628 (565h33m29.198056767s) [noise=true , address=145.79.227.18:4005 , piece_id=27PBR6OCJYCV76LXCWGZ4RMHYHYELU7UUQWK77ONLAKO7OZP5RJA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.845900639 (565h33m29.198685767s) [noise=true , address=45.91.116.4:4939 , piece_id=X6PJQNIPP5DGU4LGUJZEXJXR7FAARPN77W5D6MBNRMHPDEPY25AA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.846045771 (565h33m29.1988309s) [noise=true , address=45.91.116.7:4010 , piece_id=CKBCDXJE3ATNSEDNH4DNFH3WL4GZMVMEXVXRLI7GRY3XFUU2WOMA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.846725577 (565h33m29.199510705s) [noise=true , address=77.90.31.19:4842 , piece_id=WOMDAGSW44HXQV7E6JQEQBZKHIYGLW25QPBRCPK2XCRT62MUAHYQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.847449616 (565h33m29.200234744s) [noise=true , address=77.90.31.16:4733 , piece_id=MHKBKDVWJ2RZQXRLMTO33LSX7YJWJ3PMZJSVRO3RXKDXP2F7CSVA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.847961646 (565h33m29.200746775s) [noise=true , address=45.91.116.11:4118 , piece_id=R6HKZQ2B5AQ4C7YHHPBON4SVHPIV2IJN7DMSY6H7BSX7WHTSPKAQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.849104080 (565h33m29.201889208s) [noise=true , address=77.90.31.12:4592 , piece_id=CKT6LHANOHVC4NHMRQ2SEEXFPQA7MHM44RKOISA45M26QAWIPDVA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.850005882 (565h33m29.202791011s) [noise=true , address=145.79.227.4:4186 , piece_id=DJEVDGISBPJJ4R37Z5RXWHVLOXZRPMMVC3PWO66H24JQHY46QAUA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.982635031 (565h33m29.335420168s) [upload=497735]: Next returned nil segment Sep 22 03:41:53.982652824 (565h33m29.335437952s) [upload=497735]: Waiting for error group managing segments... Sep 22 03:41:53.990044095 (565h33m29.342829232s) [noise=true , address=45.91.116.11:4118 , piece_id=R6HKZQ2B5AQ4C7YHHPBON4SVHPIV2IJN7DMSY6H7BSX7WHTSPKAQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.990531790 (565h33m29.343316918s) [noise=true , address=45.91.116.13:4154 , piece_id=M2Q6RNA3AKZUBYFE6OMIZOFTRX7ASU7D3SPCVUGUDLMIQ7TZIZWA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991047046 (565h33m29.343832184s) [noise=true , address=45.91.116.9:4261 , piece_id=KXFVZ2ES3MGT26OJRKEHYW5DPGPMEFUF2WTEZXEQFXZBLHEDILKA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991466263 (565h33m29.344251392s) [noise=true , address=45.91.116.7:4009 , piece_id=3K2HO6HF3KR7PIJRDABE7FW4A2FVEKHNR3ANPMU7MWVILDCKYX6Q, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991769693 (565h33m29.344554831s) [noise=true , address=45.91.116.4:4939 , piece_id=X6PJQNIPP5DGU4LGUJZEXJXR7FAARPN77W5D6MBNRMHPDEPY25AA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.992458005 (565h33m29.345243133s) [noise=true , address=77.90.31.19:4842 , piece_id=WOMDAGSW44HXQV7E6JQEQBZKHIYGLW25QPBRCPK2XCRT62MUAHYQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.992505003 (565h33m29.345290131s) [noise=true , address=45.91.116.7:4010 , piece_id=CKBCDXJE3ATNSEDNH4DNFH3WL4GZMVMEXVXRLI7GRY3XFUU2WOMA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.993159922 (565h33m29.34594505s) [noise=true , address=77.90.31.12:4592 , piece_id=CKT6LHANOHVC4NHMRQ2SEEXFPQA7MHM44RKOISA45M26QAWIPDVA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.993274718 (565h33m29.346059845s) [noise=true , address=45.91.116.8:4047 , piece_id=GWNXFH4LDIAYWDKNKVONYRFVTTKBQHLR72ZHHFZ7M2YDXCBXLRIQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.994692838 (565h33m29.347477966s) [noise=true , address=77.90.31.21:4878 , piece_id=TPRVB7U4ZNR7BOYT4BC22JIDHV3IHBTWM76R25UK5JRYR4XUSI7A, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.994841236 (565h33m29.347626365s) [noise=true , address=77.90.31.13:4628 , piece_id=BLD55WPW4WBWJ6GWBBJDXWIYRQNPCUBA3N5UFZDT4WFKMDJOKP3Q, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.995361182 (565h33m29.348146311s) [noise=true , address=77.90.31.16:4733 , piece_id=MHKBKDVWJ2RZQXRLMTO33LSX7YJWJ3PMZJSVRO3RXKDXP2F7CSVA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:42:23.809279587 (565h33m59.162064714s) [noise=true , address=145.79.227.4:4186 , piece_id=DJEVDGISBPJJ4R37Z5RXWHVLOXZRPMMVC3PWO66H24JQHY46QAUA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:42:23.809421593 (565h33m59.162206721s) [noise=true , address=145.79.227.12:4765 , piece_id=UJQMQKN3YUVQ2AQNXBPYV2ZUUKWY7Z4F6FHG4DXKI6ARMA2TFIXQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:42:23.809560264 (565h33m59.162345402s) [noise=true , address=145.79.227.6:4259 , piece_id=WORJPEIFUREEUUCOV3P4RPFEQLXGHSUD4ZBLNLN2OVJDUAHVTDHA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:42:23.809598205 (565h33m59.162383333s) [noise=true , address=145.79.227.18:4005 , piece_id=27PBR6OCJYCV76LXCWGZ4RMHYHYELU7UUQWK77ONLAKO7OZP5RJA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; context deadline exceeded Sep 22 03:42:23.809711137 (565h33m59.162496265s) [seg_pos={0 0}, upload=497735]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:42:23.825581299 (565h33m59.178366426s) [upload=497735]: Done uploading segments... Sep 22 03:42:23.825600274 (565h33m59.178385412s) [upload=497735]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 447.635571ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 2.189527545s FAIL (2025-09-22 03:43:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.189552482s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..........XX.....X..X...X..........X...X.XX...XXX...X.......] duration: 604.504728ms FAIL (2025-09-22 03:43:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 604.522241ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X....................................X..X...X...X.X.....XX..] duration: 494.818802ms FAIL (2025-09-22 03:43:10 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 03:41:53.821360598 (565h33m29.174145726s) [upload=497735]: Starting upload Sep 22 03:41:53.821432342 (565h33m29.174217471s) [upload=497735]: Done starting upload Sep 22 03:41:53.821573697 (565h33m29.174358826s) [upload=497735]: Uploading segments... Sep 22 03:41:53.821895693 (565h33m29.174680821s) [upload=497735]: Got next segment. Inline: false Sep 22 03:41:53.821989369 (565h33m29.174774496s) [upload=497735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 03:41:53.821995941 (565h33m29.174781069s) [upload=497735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 03:41:53.843161018 (565h33m29.195946156s) [seg_pos={0 0}, upload=497735]: Begin upload segment... Sep 22 03:41:53.843219267 (565h33m29.196004405s) [seg_pos={0 0}, upload=497735]: Done begin upload segment. Sep 22 03:41:53.843260845 (565h33m29.196045973s) [noise=true , address=45.91.116.8:4047 , piece_id=GWNXFH4LDIAYWDKNKVONYRFVTTKBQHLR72ZHHFZ7M2YDXCBXLRIQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.843270974 (565h33m29.196056102s) [noise=true , address=45.91.116.7:4009 , piece_id=3K2HO6HF3KR7PIJRDABE7FW4A2FVEKHNR3ANPMU7MWVILDCKYX6Q, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.843790710 (565h33m29.196575838s) [noise=true , address=145.79.227.6:4259 , piece_id=WORJPEIFUREEUUCOV3P4RPFEQLXGHSUD4ZBLNLN2OVJDUAHVTDHA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844019850 (565h33m29.196804988s) [noise=true , address=45.91.116.13:4154 , piece_id=M2Q6RNA3AKZUBYFE6OMIZOFTRX7ASU7D3SPCVUGUDLMIQ7TZIZWA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844360168 (565h33m29.197145296s) [noise=true , address=145.79.227.12:4765 , piece_id=UJQMQKN3YUVQ2AQNXBPYV2ZUUKWY7Z4F6FHG4DXKI6ARMA2TFIXQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844472439 (565h33m29.197257576s) [noise=true , address=77.90.31.21:4878 , piece_id=TPRVB7U4ZNR7BOYT4BC22JIDHV3IHBTWM76R25UK5JRYR4XUSI7A, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844820201 (565h33m29.19760533s) [noise=true , address=77.90.31.13:4628 , piece_id=BLD55WPW4WBWJ6GWBBJDXWIYRQNPCUBA3N5UFZDT4WFKMDJOKP3Q, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.844922043 (565h33m29.197707171s) [noise=true , address=45.91.116.9:4261 , piece_id=KXFVZ2ES3MGT26OJRKEHYW5DPGPMEFUF2WTEZXEQFXZBLHEDILKA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.845271628 (565h33m29.198056767s) [noise=true , address=145.79.227.18:4005 , piece_id=27PBR6OCJYCV76LXCWGZ4RMHYHYELU7UUQWK77ONLAKO7OZP5RJA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.845900639 (565h33m29.198685767s) [noise=true , address=45.91.116.4:4939 , piece_id=X6PJQNIPP5DGU4LGUJZEXJXR7FAARPN77W5D6MBNRMHPDEPY25AA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.846045771 (565h33m29.1988309s) [noise=true , address=45.91.116.7:4010 , piece_id=CKBCDXJE3ATNSEDNH4DNFH3WL4GZMVMEXVXRLI7GRY3XFUU2WOMA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.846725577 (565h33m29.199510705s) [noise=true , address=77.90.31.19:4842 , piece_id=WOMDAGSW44HXQV7E6JQEQBZKHIYGLW25QPBRCPK2XCRT62MUAHYQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.847449616 (565h33m29.200234744s) [noise=true , address=77.90.31.16:4733 , piece_id=MHKBKDVWJ2RZQXRLMTO33LSX7YJWJ3PMZJSVRO3RXKDXP2F7CSVA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.847961646 (565h33m29.200746775s) [noise=true , address=45.91.116.11:4118 , piece_id=R6HKZQ2B5AQ4C7YHHPBON4SVHPIV2IJN7DMSY6H7BSX7WHTSPKAQ, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.849104080 (565h33m29.201889208s) [noise=true , address=77.90.31.12:4592 , piece_id=CKT6LHANOHVC4NHMRQ2SEEXFPQA7MHM44RKOISA45M26QAWIPDVA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.850005882 (565h33m29.202791011s) [noise=true , address=145.79.227.4:4186 , piece_id=DJEVDGISBPJJ4R37Z5RXWHVLOXZRPMMVC3PWO66H24JQHY46QAUA, seg_pos={0 0}, upload=497735]: Uploading piece... Sep 22 03:41:53.982635031 (565h33m29.335420168s) [upload=497735]: Next returned nil segment Sep 22 03:41:53.982652824 (565h33m29.335437952s) [upload=497735]: Waiting for error group managing segments... Sep 22 03:41:53.990044095 (565h33m29.342829232s) [noise=true , address=45.91.116.11:4118 , piece_id=R6HKZQ2B5AQ4C7YHHPBON4SVHPIV2IJN7DMSY6H7BSX7WHTSPKAQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.990531790 (565h33m29.343316918s) [noise=true , address=45.91.116.13:4154 , piece_id=M2Q6RNA3AKZUBYFE6OMIZOFTRX7ASU7D3SPCVUGUDLMIQ7TZIZWA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991047046 (565h33m29.343832184s) [noise=true , address=45.91.116.9:4261 , piece_id=KXFVZ2ES3MGT26OJRKEHYW5DPGPMEFUF2WTEZXEQFXZBLHEDILKA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991466263 (565h33m29.344251392s) [noise=true , address=45.91.116.7:4009 , piece_id=3K2HO6HF3KR7PIJRDABE7FW4A2FVEKHNR3ANPMU7MWVILDCKYX6Q, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.991769693 (565h33m29.344554831s) [noise=true , address=45.91.116.4:4939 , piece_id=X6PJQNIPP5DGU4LGUJZEXJXR7FAARPN77W5D6MBNRMHPDEPY25AA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.992458005 (565h33m29.345243133s) [noise=true , address=77.90.31.19:4842 , piece_id=WOMDAGSW44HXQV7E6JQEQBZKHIYGLW25QPBRCPK2XCRT62MUAHYQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.992505003 (565h33m29.345290131s) [noise=true , address=45.91.116.7:4010 , piece_id=CKBCDXJE3ATNSEDNH4DNFH3WL4GZMVMEXVXRLI7GRY3XFUU2WOMA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.993159922 (565h33m29.34594505s) [noise=true , address=77.90.31.12:4592 , piece_id=CKT6LHANOHVC4NHMRQ2SEEXFPQA7MHM44RKOISA45M26QAWIPDVA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.993274718 (565h33m29.346059845s) [noise=true , address=45.91.116.8:4047 , piece_id=GWNXFH4LDIAYWDKNKVONYRFVTTKBQHLR72ZHHFZ7M2YDXCBXLRIQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.994692838 (565h33m29.347477966s) [noise=true , address=77.90.31.21:4878 , piece_id=TPRVB7U4ZNR7BOYT4BC22JIDHV3IHBTWM76R25UK5JRYR4XUSI7A, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.994841236 (565h33m29.347626365s) [noise=true , address=77.90.31.13:4628 , piece_id=BLD55WPW4WBWJ6GWBBJDXWIYRQNPCUBA3N5UFZDT4WFKMDJOKP3Q, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:41:53.995361182 (565h33m29.348146311s) [noise=true , address=77.90.31.16:4733 , piece_id=MHKBKDVWJ2RZQXRLMTO33LSX7YJWJ3PMZJSVRO3RXKDXP2F7CSVA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: <nil> Sep 22 03:42:23.809279587 (565h33m59.162064714s) [noise=true , address=145.79.227.4:4186 , piece_id=DJEVDGISBPJJ4R37Z5RXWHVLOXZRPMMVC3PWO66H24JQHY46QAUA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: rpc: dial tcp 145.79.227.4:4186: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:42:23.809421593 (565h33m59.162206721s) [noise=true , address=145.79.227.12:4765 , piece_id=UJQMQKN3YUVQ2AQNXBPYV2ZUUKWY7Z4F6FHG4DXKI6ARMA2TFIXQ, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: rpc: dial tcp 145.79.227.12:4765: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 03:42:23.809560264 (565h33m59.162345402s) [noise=true , address=145.79.227.6:4259 , piece_id=WORJPEIFUREEUUCOV3P4RPFEQLXGHSUD4ZBLNLN2OVJDUAHVTDHA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 03:42:23.809598205 (565h33m59.162383333s) [noise=true , address=145.79.227.18:4005 , piece_id=27PBR6OCJYCV76LXCWGZ4RMHYHYELU7UUQWK77ONLAKO7OZP5RJA, seg_pos={0 0}, upload=497735]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: i/o timeout; context deadline exceeded Sep 22 03:42:23.809711137 (565h33m59.162496265s) [seg_pos={0 0}, upload=497735]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 03:42:23.825581299 (565h33m59.178366426s) [upload=497735]: Done uploading segments... Sep 22 03:42:23.825600274 (565h33m59.178385412s) [upload=497735]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 494.841886ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)