Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.001008462s FAIL (2025-08-07 21:16:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001026416s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 855.627455ms FAIL (2025-08-07 21:16:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 855.650838ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [....................................................XX..XXXX] duration: 30.001256314s FAIL (2025-08-07 21:16:32 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001268015s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.001094399s FAIL (2025-08-07 21:16:24 UTC) operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== , upload=4938]: Uploading piece... Aug 7 21:13:09.735341183 (4h46m23.322397425s) [noise=true , address=104.36.82.13:4185 , piece_id=JRWMK5KTTCJIL5JJ4YVJ3GHS6BYCZP3CRWM4AN4ZL2UU4HJADBSQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735661972 (4h46m23.322718213s) [noise=true , address=104.36.82.7:4910 , piece_id=HFOIPKET4SKDQ4LRX54T7VOA7V2Y46MZ6RH5DED63ZXTTDYZTHQA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735696867 (4h46m23.322753117s) [noise=true , address=104.36.82.12:4151 , piece_id=OIWOXCCCVAW3B5ISSLNWASC6TD2WSWC4J3EJMF554OHVQMCZUIWQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735736471 (4h46m23.322792711s) [noise=true , address=77.90.31.17:4772 , piece_id=QUJ2AZMAXL2FFEWVASVIN734LGSMBFRXHX5KORJLQNGIXNN64ZVA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736212920 (4h46m23.323269172s) [noise=true , address=104.36.82.2:4729 , piece_id=KQOVEHNICVG5FWPV2B7CP3U6SMP56T7PRNZOG65QDAX2MP4SEN7Q, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736236384 (4h46m23.323292625s) [noise=true , address=77.90.31.18:4807 , piece_id=GYJGSHU2PC4DM5VA6SLHGM7S45KOLLBLBTMKD6LT5LMAEMHKGK5Q, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736629768 (4h46m23.323686009s) [noise=true , address=104.36.82.13:4187 , piece_id=J3E4KCSQZDYKXQMOTRSUUUIU2SZIRFBM7WD32FCFJL7V7DILAQXA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736782593 (4h46m23.323838835s) [noise=true , address=45.91.116.12:4225 , piece_id=EPHX3ZYTEBJUTQP56HQSL4GPW5SBMC7KDNSR6BD7CRQTMCDUZ2DQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.737971863 (4h46m23.325028114s) [noise=true , address=104.36.82.9:4006 , piece_id=EH5ZUHUO55F7YYVD2TSWXDIUX65GY2VVVMBW5DOFRADL3WO3ZQGA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.841271227 (4h46m23.428327438s) [upload=4938]: Next returned nil segment Aug 7 21:13:09.841281055 (4h46m23.428337266s) [upload=4938]: Waiting for error group managing segments... Aug 7 21:13:09.853631449 (4h46m23.440687659s) [noise=true , address=45.91.116.4:4937 , piece_id=HUS3E4LN6CYOXDY2DIEK4OATMZPZADDVIQO3CM3L5QEMB567P3KA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.854204258 (4h46m23.441260499s) [noise=true , address=45.91.116.12:4225 , piece_id=EPHX3ZYTEBJUTQP56HQSL4GPW5SBMC7KDNSR6BD7CRQTMCDUZ2DQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.854658546 (4h46m23.441714757s) [noise=true , address=45.91.116.14:4191 , piece_id=23HFDLS3OMLHX4EBBB3N6ORXPOJGZ4TSLXD622Z5QW57I7F33YSQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.855360527 (4h46m23.442416778s) [noise=true , address=77.90.31.18:4807 , piece_id=GYJGSHU2PC4DM5VA6SLHGM7S45KOLLBLBTMKD6LT5LMAEMHKGK5Q, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.855562183 (4h46m23.442618394s) [noise=true , address=104.36.82.13:4185 , piece_id=JRWMK5KTTCJIL5JJ4YVJ3GHS6BYCZP3CRWM4AN4ZL2UU4HJADBSQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.856428770 (4h46m23.443485021s) [noise=true , address=77.90.31.19:4841 , piece_id=AKJV4UNYIXKLSI4Y5STPWBXJ3AFZERBSVCIER53F23KL5WABQFTQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.856590482 (4h46m23.443646684s) [noise=true , address=104.36.82.2:4729 , piece_id=KQOVEHNICVG5FWPV2B7CP3U6SMP56T7PRNZOG65QDAX2MP4SEN7Q, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.858568354 (4h46m23.445624565s) [noise=true , address=45.91.116.13:4155 , piece_id=BXSMK5DW67MELF2KN6CIDHJ5HOSF7MZBRVSA7WQ45UV4SLI5XHWQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.858850201 (4h46m23.445906441s) [noise=true , address=77.90.31.16:4733 , piece_id=EMDHMK2PGYUT3SUXOVNEKLRHOH3XY4WNXON4OXTERRMZI43GCROA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.859375651 (4h46m23.446431862s) [noise=true , address=77.90.31.17:4772 , piece_id=QUJ2AZMAXL2FFEWVASVIN734LGSMBFRXHX5KORJLQNGIXNN64ZVA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.859727598 (4h46m23.446783819s) [noise=true , address=104.36.82.13:4187 , piece_id=J3E4KCSQZDYKXQMOTRSUUUIU2SZIRFBM7WD32FCFJL7V7DILAQXA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862181970 (4h46m23.44923821s) [noise=true , address=104.36.82.12:4151 , piece_id=OIWOXCCCVAW3B5ISSLNWASC6TD2WSWC4J3EJMF554OHVQMCZUIWQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862286064 (4h46m23.449342285s) [noise=true , address=104.36.82.7:4910 , piece_id=HFOIPKET4SKDQ4LRX54T7VOA7V2Y46MZ6RH5DED63ZXTTDYZTHQA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862295481 (4h46m23.449351692s) [seg_pos={0 0}, upload=4938]: Segment reached optimal threshold of 13 pieces. Aug 7 21:13:09.862540559 (4h46m23.44959677s) [noise=true , address=104.36.82.9:4006 , piece_id=EH5ZUHUO55F7YYVD2TSWXDIUX65GY2VVVMBW5DOFRADL3WO3ZQGA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection; write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection Aug 7 21:13:09.862594711 (4h46m23.449650932s) [noise=true , address=104.36.82.5:4839 , piece_id=JFXBI2UEF5IPMF3NFKEX4CBRRNCIBJCSLLSVKCTFRTYRD3U4TLLQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection; write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection Aug 7 21:13:09.862660843 (4h46m23.449717085s) [noise=true , address=104.36.82.15:4259 , piece_id=AZDG4LGBEPEVUA4KZH24JWD2BCFEUA2IGBFWC57N4D5WVXBJ27PA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection; write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection Aug 7 21:13:09.862727478 (4h46m23.449783718s) [seg_pos={0 0}, upload=4938]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:13:09.862892626 (4h46m23.449948867s) [upload=4938]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:13:09.862899970 (4h46m23.449956211s) [upload=4938]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:09.912447959 (4h46m23.499504169s) [upload=4938]: Done uploading segments... Aug 7 21:13:09.912480760 (4h46m23.499536981s) [upload=4938]: Upload finished. err: <nil> Aug 7 21:14:19.537010001 (4h47m33.124066212s) [upload=4958]: Starting upload Aug 7 21:14:19.537096253 (4h47m33.124152504s) [upload=4958]: Done starting upload Aug 7 21:14:19.537202491 (4h47m33.124258732s) [upload=4958]: Uploading segments... Aug 7 21:14:19.537385743 (4h47m33.124441964s) [upload=4958]: Got next segment. Inline: false Aug 7 21:14:19.537463918 (4h47m33.124520139s) [upload=4958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:19.537469729 (4h47m33.12452595s) [upload=4958]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:49.522961209 (4h48m3.11001742s) [upload=4958]: Next segment err: context deadline exceeded Aug 7 21:14:49.523117610 (4h48m3.110173821s) [upload=4958]: Done uploading segments... Aug 7 21:14:49.523153057 (4h48m3.110209278s) [upload=4958]: Upload finished. err: context deadline exceeded Aug 7 21:15:15.778236986 (4h48m29.365293237s) [upload=4971]: Starting upload Aug 7 21:15:15.778363202 (4h48m29.365419443s) [upload=4971]: Done starting upload Aug 7 21:15:15.778550181 (4h48m29.365606442s) [upload=4971]: Uploading segments... Aug 7 21:15:15.778800839 (4h48m29.36585709s) [upload=4971]: Got next segment. Inline: false Aug 7 21:15:15.778897088 (4h48m29.36595334s) [upload=4971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:15.778908650 (4h48m29.365964891s) [upload=4971]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:15:45.754152630 (4h48m59.341208851s) [upload=4971]: Next segment err: context deadline exceeded Aug 7 21:15:45.754250592 (4h48m59.341306803s) [upload=4971]: Done uploading segments... Aug 7 21:15:45.754272974 (4h48m59.341329186s) [upload=4971]: Upload finished. err: context deadline exceeded Aug 7 21:15:54.388794729 (4h49m7.975850951s) [upload=4981]: Starting upload Aug 7 21:15:54.388846236 (4h49m7.975902457s) [upload=4981]: Done starting upload Aug 7 21:15:54.388936303 (4h49m7.975992524s) [upload=4981]: Uploading segments... Aug 7 21:15:54.389106811 (4h49m7.976163032s) [upload=4981]: Got next segment. Inline: false Aug 7 21:15:54.389136207 (4h49m7.976192428s) [upload=4981]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:54.389140785 (4h49m7.976197006s) [upload=4981]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:16:24.377635868 (4h49m37.96469211s) [upload=4981]: Next segment err: context deadline exceeded Aug 7 21:16:24.377706209 (4h49m37.96476246s) [upload=4981]: Done uploading segments... Aug 7 21:16:24.377728742 (4h49m37.964784992s) [upload=4981]: Upload finished. err: context deadline exceeded Duration: 30.00111621s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.001523469s FAIL (2025-08-07 21:16:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 6576423 (4h41m6.893632675s) [upload=4846]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:07:53.372716776 (4h41m6.959773018s) [upload=4846]: Done uploading segments... Aug 7 21:07:53.372735220 (4h41m6.959791462s) [upload=4846]: Upload finished. err: <nil> Aug 7 21:08:39.614384111 (4h41m53.201440353s) [upload=4858]: Starting upload Aug 7 21:08:39.614474791 (4h41m53.201531031s) [upload=4858]: Done starting upload Aug 7 21:08:39.614542067 (4h41m53.201598308s) [upload=4858]: Uploading segments... Aug 7 21:08:39.614579557 (4h41m53.201635799s) [upload=4858]: Got next segment. Inline: true Aug 7 21:08:39.614585047 (4h41m53.201641289s) [upload=4858]: Waiting for error group managing segments... Aug 7 21:08:39.614629911 (4h41m53.201686161s) [upload=4858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:08:39.614635952 (4h41m53.201692193s) [upload=4858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:08:39.614639619 (4h41m53.20169586s) [upload=4858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:08:39.738955514 (4h41m53.326011754s) [upload=4858]: Done uploading segments... Aug 7 21:08:39.738974128 (4h41m53.32603037s) [upload=4858]: Upload finished. err: <nil> Aug 7 21:09:18.312602069 (4h42m31.899658319s) [upload=4869]: Starting upload Aug 7 21:09:18.312710330 (4h42m31.899766582s) [upload=4869]: Done starting upload Aug 7 21:09:18.312805559 (4h42m31.89986181s) [upload=4869]: Uploading segments... Aug 7 21:09:18.312872103 (4h42m31.899928344s) [upload=4869]: Got next segment. Inline: true Aug 7 21:09:18.312879376 (4h42m31.899935618s) [upload=4869]: Waiting for error group managing segments... Aug 7 21:09:18.312928889 (4h42m31.89998514s) [upload=4869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:09:18.312934650 (4h42m31.8999909s) [upload=4869]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:09:18.312938246 (4h42m31.899994497s) [upload=4869]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:09:18.367751803 (4h42m31.954808044s) [upload=4869]: Done uploading segments... Aug 7 21:09:18.367774827 (4h42m31.954831078s) [upload=4869]: Upload finished. err: <nil> Aug 7 21:10:26.543295840 (4h43m40.130352072s) [upload=4887]: Starting upload Aug 7 21:10:26.543341496 (4h43m40.130397717s) [upload=4887]: Done starting upload Aug 7 21:10:26.543385698 (4h43m40.130441919s) [upload=4887]: Uploading segments... Aug 7 21:10:26.543414672 (4h43m40.130470883s) [upload=4887]: Got next segment. Inline: true Aug 7 21:10:26.543418339 (4h43m40.13047455s) [upload=4887]: Waiting for error group managing segments... Aug 7 21:10:26.543446882 (4h43m40.130503103s) [upload=4887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:26.543450790 (4h43m40.130507011s) [upload=4887]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:26.543453054 (4h43m40.130509265s) [upload=4887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:26.611321150 (4h43m40.198377371s) [upload=4887]: Done uploading segments... Aug 7 21:10:26.611339073 (4h43m40.198395294s) [upload=4887]: Upload finished. err: <nil> Aug 7 21:10:58.699165641 (4h44m12.286221892s) [upload=4897]: Starting upload Aug 7 21:10:58.699245781 (4h44m12.286302022s) [upload=4897]: Done starting upload Aug 7 21:10:58.699314669 (4h44m12.286370911s) [upload=4897]: Uploading segments... Aug 7 21:10:58.699373410 (4h44m12.286429651s) [upload=4897]: Got next segment. Inline: true Aug 7 21:10:58.699380724 (4h44m12.286436964s) [upload=4897]: Waiting for error group managing segments... Aug 7 21:10:58.699426338 (4h44m12.286482579s) [upload=4897]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:58.699432380 (4h44m12.28648862s) [upload=4897]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:58.699436066 (4h44m12.286492317s) [upload=4897]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:58.775720472 (4h44m12.362776712s) [upload=4897]: Done uploading segments... Aug 7 21:10:58.775736111 (4h44m12.362792352s) [upload=4897]: Upload finished. err: <nil> Aug 7 21:11:31.613155564 (4h44m45.200211785s) [upload=4908]: Starting upload Aug 7 21:11:31.613232477 (4h44m45.200288698s) [upload=4908]: Done starting upload Aug 7 21:11:31.613307799 (4h44m45.20036401s) [upload=4908]: Uploading segments... Aug 7 21:11:31.613347051 (4h44m45.200403272s) [upload=4908]: Got next segment. Inline: true Aug 7 21:11:31.613352932 (4h44m45.200409153s) [upload=4908]: Waiting for error group managing segments... Aug 7 21:11:31.613404179 (4h44m45.2004604s) [upload=4908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:31.613410591 (4h44m45.200466842s) [upload=4908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:31.613414508 (4h44m45.200470729s) [upload=4908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:31.682826735 (4h44m45.269882977s) [upload=4908]: Done uploading segments... Aug 7 21:11:31.682853265 (4h44m45.269909476s) [upload=4908]: Upload finished. err: <nil> Aug 7 21:11:51.754258397 (4h45m5.341314618s) [upload=4915]: Starting upload Aug 7 21:11:51.754330331 (4h45m5.341386542s) [upload=4915]: Done starting upload Aug 7 21:11:51.754383972 (4h45m5.341440193s) [upload=4915]: Uploading segments... Aug 7 21:11:51.754417083 (4h45m5.341473294s) [upload=4915]: Got next segment. Inline: true Aug 7 21:11:51.754420630 (4h45m5.34147684s) [upload=4915]: Waiting for error group managing segments... Aug 7 21:11:51.754453131 (4h45m5.341509352s) [upload=4915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:51.754457108 (4h45m5.341513329s) [upload=4915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:51.754460725 (4h45m5.341516936s) [upload=4915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:51.830658438 (4h45m5.417714649s) [upload=4915]: Done uploading segments... Aug 7 21:11:51.830673094 (4h45m5.417729315s) [upload=4915]: Upload finished. err: <nil> Aug 7 21:12:54.778494646 (4h46m8.365550856s) [upload=4932]: Starting upload Aug 7 21:12:54.778560719 (4h46m8.36561693s) [upload=4932]: Done starting upload Aug 7 21:12:54.778608638 (4h46m8.365664859s) [upload=4932]: Uploading segments... Aug 7 21:12:54.778637322 (4h46m8.365693543s) [upload=4932]: Got next segment. Inline: true Aug 7 21:12:54.778640999 (4h46m8.36569722s) [upload=4932]: Waiting for error group managing segments... Aug 7 21:12:54.778677527 (4h46m8.365733739s) [upload=4932]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:54.778681655 (4h46m8.365737876s) [upload=4932]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:54.778684771 (4h46m8.365740983s) [upload=4932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:54.868239378 (4h46m8.455295599s) [upload=4932]: Done uploading segments... Aug 7 21:12:54.868253865 (4h46m8.455310086s) [upload=4932]: Upload finished. err: <nil> Aug 7 21:13:23.238906839 (4h46m36.82596306s) [upload=4942]: Starting upload Aug 7 21:13:23.238959356 (4h46m36.826015567s) [upload=4942]: Done starting upload Aug 7 21:13:23.239011013 (4h46m36.826067234s) [upload=4942]: Uploading segments... Aug 7 21:13:23.239039727 (4h46m36.826095937s) [upload=4942]: Got next segment. Inline: true Aug 7 21:13:23.239043483 (4h46m36.826099695s) [upload=4942]: Waiting for error group managing segments... Aug 7 21:13:23.239098526 (4h46m36.826154737s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:23.239104707 (4h46m36.826160919s) [upload=4942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:23.239108474 (4h46m36.826164696s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:23.309357825 (4h46m36.896414127s) [upload=4942]: Done uploading segments... Aug 7 21:13:23.309383974 (4h46m36.896440216s) [upload=4942]: Upload finished. err: <nil> Aug 7 21:14:18.294119361 (4h47m31.881175572s) [upload=4957]: Starting upload Aug 7 21:14:18.294187468 (4h47m31.881243679s) [upload=4957]: Done starting upload Aug 7 21:14:18.294261516 (4h47m31.881317737s) [upload=4957]: Uploading segments... Aug 7 21:14:18.294305068 (4h47m31.881361289s) [upload=4957]: Got next segment. Inline: true Aug 7 21:14:18.294311409 (4h47m31.881367621s) [upload=4957]: Waiting for error group managing segments... Aug 7 21:14:18.294349491 (4h47m31.881405712s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.294354059 (4h47m31.88141028s) [upload=4957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.294356643 (4h47m31.881412854s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:18.280461255 (4h48m31.867517496s) [upload=4957]: Done uploading segments... Aug 7 21:15:18.280484308 (4h48m31.867540559s) [upload=4957]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:18.387536247 (4h48m31.974592488s) [upload=4974]: Starting upload Aug 7 21:15:18.387607770 (4h48m31.974664011s) [upload=4974]: Done starting upload Aug 7 21:15:18.387666109 (4h48m31.974722361s) [upload=4974]: Uploading segments... Aug 7 21:15:18.387706996 (4h48m31.974763236s) [upload=4974]: Got next segment. Inline: true Aug 7 21:15:18.387712766 (4h48m31.974769007s) [upload=4974]: Waiting for error group managing segments... Aug 7 21:15:18.387755796 (4h48m31.974812038s) [upload=4974]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:18.387761227 (4h48m31.974817468s) [upload=4974]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:18.387764853 (4h48m31.974821095s) [upload=4974]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:18.373970870 (4h49m31.961027121s) [upload=4974]: Done uploading segments... Aug 7 21:16:18.374035110 (4h49m31.961091352s) [upload=4974]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1m0.001542604s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000771493s FAIL (2025-08-07 21:16:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000795389s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration: 30.000971236s FAIL (2025-08-07 21:16:10 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 7 21:07:44.986648167 (4h40m58.573704407s) [upload=4843]: Done uploading segments... Aug 7 21:07:44.986675408 (4h40m58.573731649s) [upload=4843]: Upload finished. err: <nil> Aug 7 21:08:51.184262427 (4h42m4.771318639s) [upload=4861]: Starting upload Aug 7 21:08:51.184314965 (4h42m4.771371176s) [upload=4861]: Done starting upload Aug 7 21:08:51.184366822 (4h42m4.771423043s) [upload=4861]: Uploading segments... Aug 7 21:08:51.184393902 (4h42m4.771450113s) [upload=4861]: Got next segment. Inline: true Aug 7 21:08:51.184397439 (4h42m4.77145365s) [upload=4861]: Waiting for error group managing segments... Aug 7 21:08:51.184429829 (4h42m4.771486041s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:08:51.184433546 (4h42m4.771489758s) [upload=4861]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:08:51.184435881 (4h42m4.771492092s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:08:51.240315229 (4h42m4.82737147s) [upload=4861]: Done uploading segments... Aug 7 21:08:51.240335075 (4h42m4.827391316s) [upload=4861]: Upload finished. err: <nil> Aug 7 21:09:58.632781085 (4h43m12.219837326s) [upload=4880]: Starting upload Aug 7 21:09:58.632867045 (4h43m12.219923286s) [upload=4880]: Done starting upload Aug 7 21:09:58.632946213 (4h43m12.220002465s) [upload=4880]: Uploading segments... Aug 7 21:09:58.632990476 (4h43m12.220046716s) [upload=4880]: Got next segment. Inline: true Aug 7 21:09:58.632996186 (4h43m12.220052437s) [upload=4880]: Waiting for error group managing segments... Aug 7 21:09:58.633044917 (4h43m12.220101159s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:09:58.633050438 (4h43m12.220106679s) [upload=4880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:09:58.633077508 (4h43m12.220133759s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:09:58.687844850 (4h43m12.274901172s) [upload=4880]: Done uploading segments... Aug 7 21:09:58.687873194 (4h43m12.274929435s) [upload=4880]: Upload finished. err: <nil> Aug 7 21:10:48.980309603 (4h44m2.567365814s) [upload=4895]: Starting upload Aug 7 21:10:48.980351321 (4h44m2.567407542s) [upload=4895]: Done starting upload Aug 7 21:10:48.980412926 (4h44m2.567469137s) [upload=4895]: Uploading segments... Aug 7 21:10:48.980468068 (4h44m2.567524309s) [upload=4895]: Got next segment. Inline: true Aug 7 21:10:48.980474410 (4h44m2.567530621s) [upload=4895]: Waiting for error group managing segments... Aug 7 21:10:48.980519745 (4h44m2.567575966s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:48.980525596 (4h44m2.567581807s) [upload=4895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:48.980529363 (4h44m2.567585584s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:49.036078074 (4h44m2.623134295s) [upload=4895]: Done uploading segments... Aug 7 21:10:49.036092611 (4h44m2.623148821s) [upload=4895]: Upload finished. err: <nil> Aug 7 21:11:10.299224768 (4h44m23.886281009s) [upload=4903]: Starting upload Aug 7 21:11:10.299304937 (4h44m23.886361178s) [upload=4903]: Done starting upload Aug 7 21:11:10.299368445 (4h44m23.886424686s) [upload=4903]: Uploading segments... Aug 7 21:11:10.299409052 (4h44m23.886465292s) [upload=4903]: Got next segment. Inline: true Aug 7 21:11:10.299414652 (4h44m23.886470893s) [upload=4903]: Waiting for error group managing segments... Aug 7 21:11:10.299460468 (4h44m23.886516709s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:10.299465988 (4h44m23.886522229s) [upload=4903]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:10.299469444 (4h44m23.886525685s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:10.444467755 (4h44m24.031523996s) [upload=4903]: Done uploading segments... Aug 7 21:11:10.444485819 (4h44m24.03154206s) [upload=4903]: Upload finished. err: <nil> Aug 7 21:12:08.543001654 (4h45m22.130057896s) [upload=4919]: Starting upload Aug 7 21:12:08.543123191 (4h45m22.130179432s) [upload=4919]: Done starting upload Aug 7 21:12:08.543216246 (4h45m22.130272487s) [upload=4919]: Uploading segments... Aug 7 21:12:08.543261810 (4h45m22.130318051s) [upload=4919]: Got next segment. Inline: true Aug 7 21:12:08.543267651 (4h45m22.130323892s) [upload=4919]: Waiting for error group managing segments... Aug 7 21:12:08.543313246 (4h45m22.130369488s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:08.543319278 (4h45m22.130375519s) [upload=4919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:08.543323075 (4h45m22.130379316s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:08.630954371 (4h45m22.218010612s) [upload=4919]: Done uploading segments... Aug 7 21:12:08.630979197 (4h45m22.218035449s) [upload=4919]: Upload finished. err: <nil> Aug 7 21:12:30.034507380 (4h45m43.62156362s) [upload=4926]: Starting upload Aug 7 21:12:30.034584302 (4h45m43.621640544s) [upload=4926]: Done starting upload Aug 7 21:12:30.034643523 (4h45m43.621699754s) [upload=4926]: Uploading segments... Aug 7 21:12:30.034681965 (4h45m43.621738206s) [upload=4926]: Got next segment. Inline: true Aug 7 21:12:30.034687645 (4h45m43.621743886s) [upload=4926]: Waiting for error group managing segments... Aug 7 21:12:30.034730766 (4h45m43.621787007s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:30.034735976 (4h45m43.621792207s) [upload=4926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:30.034739642 (4h45m43.621795883s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:00.022476261 (4h46m13.609532503s) [upload=4926]: Done uploading segments... Aug 7 21:13:00.022497211 (4h46m13.609553452s) [upload=4926]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:13:15.702096720 (4h46m29.289152941s) [upload=4939]: Starting upload Aug 7 21:13:15.702154518 (4h46m29.289210739s) [upload=4939]: Done starting upload Aug 7 21:13:15.702196716 (4h46m29.289252928s) [upload=4939]: Uploading segments... Aug 7 21:13:15.702224368 (4h46m29.289280589s) [upload=4939]: Got next segment. Inline: true Aug 7 21:13:15.702228446 (4h46m29.289284657s) [upload=4939]: Waiting for error group managing segments... Aug 7 21:13:15.702261458 (4h46m29.289317669s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:15.702265306 (4h46m29.289321527s) [upload=4939]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:15.702267880 (4h46m29.289324101s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:45.689162551 (4h46m59.276218792s) [upload=4939]: Done uploading segments... Aug 7 21:13:45.689208046 (4h46m59.276264277s) [upload=4939]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:13.522007492 (4h47m27.109063713s) [upload=4955]: Starting upload Aug 7 21:14:13.522102209 (4h47m27.10915843s) [upload=4955]: Done starting upload Aug 7 21:14:13.522154938 (4h47m27.109211179s) [upload=4955]: Uploading segments... Aug 7 21:14:13.522235488 (4h47m27.109291729s) [upload=4955]: Got next segment. Inline: true Aug 7 21:14:13.522242682 (4h47m27.109298922s) [upload=4955]: Waiting for error group managing segments... Aug 7 21:14:13.522293467 (4h47m27.109349708s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:13.522299518 (4h47m27.109355759s) [upload=4955]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:13.522303275 (4h47m27.109359516s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:43.507883903 (4h47m57.094940144s) [upload=4955]: Done uploading segments... Aug 7 21:14:43.507905914 (4h47m57.094962156s) [upload=4955]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:54.825501522 (4h48m8.412557734s) [upload=4964]: Starting upload Aug 7 21:14:54.825580730 (4h48m8.412636951s) [upload=4964]: Done starting upload Aug 7 21:14:54.825629411 (4h48m8.412685632s) [upload=4964]: Uploading segments... Aug 7 21:14:54.825660400 (4h48m8.41271661s) [upload=4964]: Got next segment. Inline: true Aug 7 21:14:54.825664547 (4h48m8.412720758s) [upload=4964]: Waiting for error group managing segments... Aug 7 21:14:54.825705873 (4h48m8.412762115s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:54.825711855 (4h48m8.412768076s) [upload=4964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:54.825715662 (4h48m8.412771883s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:24.812961328 (4h48m38.400017569s) [upload=4964]: Done uploading segments... Aug 7 21:15:24.812988770 (4h48m38.40004501s) [upload=4964]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:40.993035983 (4h48m54.580092194s) [upload=4978]: Starting upload Aug 7 21:15:40.993105403 (4h48m54.580161614s) [upload=4978]: Done starting upload Aug 7 21:15:40.993154163 (4h48m54.580210375s) [upload=4978]: Uploading segments... Aug 7 21:15:40.993189519 (4h48m54.580245741s) [upload=4978]: Got next segment. Inline: true Aug 7 21:15:40.993194319 (4h48m54.58025054s) [upload=4978]: Waiting for error group managing segments... Aug 7 21:15:40.993226930 (4h48m54.580283151s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:40.993231167 (4h48m54.580287379s) [upload=4978]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:15:40.993234264 (4h48m54.580290474s) [upload=4978]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:16:10.982081529 (4h49m24.569137771s) [upload=4978]: Done uploading segments... Aug 7 21:16:10.982104473 (4h49m24.569160713s) [upload=4978]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000990673s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 5 out of 60 is failed (threshold=3) [.....................................................XX..XXX] duration: 30.000543087s FAIL (2025-08-07 21:16:02 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000566642s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.000681733s FAIL (2025-08-07 21:16:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== piece_id=ZVVOBDRSIRLKPAOF23RMYRBXRNCNMQF3TJXYAW77ABOTGYUMXGHQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.166323508 (4h46m37.753379719s) [noise=true , address=104.36.82.62:4427 , piece_id=QMUVKOOD3THFKQAKBI3RQWH7JOP34MNNDOMKUXXBF4ZRVSYUJHJQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.166596959 (4h46m37.75365318s) [noise=true , address=104.36.82.20:4444 , piece_id=N3Z2NAQO5LHBPSABVMX4UBAAGUDDGDZCYGWX3NXWP6LUL5VJ74BA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.166809967 (4h46m37.753866178s) [noise=true , address=79.127.205.233:4643 , piece_id=IRDF6ZKJUCKCPSKAOCHW77H7O5ORLI43QULF2KWJ4O6SJTP373BQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.167018215 (4h46m37.754074437s) [noise=true , address=104.36.82.11:4130 , piece_id=434EKDAN537AQFR342ZKLLENUIZAEK4PL2UTXDRJU7NJJ5HYHLUQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.167316922 (4h46m37.754373133s) [noise=true , address=104.36.82.23:4457 , piece_id=3LDUSHS23Z6ZBTDHVGVNUZRFA3H2XDC6B243ISKKCYJJG3PCRUBA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.167677125 (4h46m37.754733346s) [noise=true , address=104.36.82.37:4680 , piece_id=3LPKEUJ5HCZL4XY7YQOI2CROEHYNBFSZV6EUFDU2WV363FWQ644Q, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.167886165 (4h46m37.754942376s) [noise=true , address=64.49.13.27:4963 , piece_id=MN6MBMOOPEIPZRW5R4NQM6AEE3AHYX33MIXOSIIZEGHDPJEKAOPA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.168145950 (4h46m37.755202161s) [noise=true , address=79.127.201.215:4354 , piece_id=4VVIMMVTX6SCVPFAODIC7JS5PVULFS2SCXMKFJKA72NXKEK62K3Q, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.168436171 (4h46m37.755492392s) [noise=true , address=104.36.82.13:4190 , piece_id=PVB3Y7KXU47SOQCZBKW4AUZ56XMLMJQ23WPM73BKHUMZKARYOCCA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.168641956 (4h46m37.755698177s) [noise=true , address=45.91.116.10:4113 , piece_id=NJYFA7PI7F62J2CSEDJUTIWHY2LW3KBSLK76TKOVJNXC3RDV35KA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.168845667 (4h46m37.755901877s) [noise=true , address=79.127.220.97:4396 , piece_id=HD3NDYSYACLTEX2OYRUDYNK5J2DREH3DTNSJSDO745Q6NCC7AXGA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.169076598 (4h46m37.756132819s) [noise=true , address=104.36.82.44:4872 , piece_id=LM4MNB6MDHTN5MVJNGJOWX5Z5AKGICGK5JCLHBQBM5IR5PD3P2BQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.169509506 (4h46m37.756565727s) [noise=true , address=104.36.82.54:4183 , piece_id=PIBOQ56366UKNWMH43GSS6TW66GFPMSJOB3UQ3TKMEYEYPKVVFOQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.169710120 (4h46m37.756766341s) [noise=true , address=104.36.82.52:4108 , piece_id=AHQKVW323AG3TIIDCPX4HYFISVIY4V2476ZQM5GXLGLQD2PJ2KVA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.260747004 (4h46m37.847803255s) [upload=4936]: Next returned nil segment Aug 7 21:13:24.260784884 (4h46m37.847841125s) [upload=4936]: Waiting for error group managing segments... Aug 7 21:13:24.282697478 (4h46m37.869753699s) [noise=true , address=104.36.82.36:4652 , piece_id=4S6GW6S5QMKXREUOZ4K7S77A5FIA5OGIS6MIPH3X5WFAIUYUTKBQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.286011234 (4h46m37.873067475s) [noise=true , address=45.91.116.10:4113 , piece_id=NJYFA7PI7F62J2CSEDJUTIWHY2LW3KBSLK76TKOVJNXC3RDV35KA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.288604064 (4h46m37.875660275s) [noise=true , address=104.36.82.23:4457 , piece_id=3LDUSHS23Z6ZBTDHVGVNUZRFA3H2XDC6B243ISKKCYJJG3PCRUBA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.289097404 (4h46m37.876153645s) [noise=true , address=104.36.82.44:4872 , piece_id=LM4MNB6MDHTN5MVJNGJOWX5Z5AKGICGK5JCLHBQBM5IR5PD3P2BQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.303818073 (4h46m37.890874284s) [noise=true , address=104.36.82.20:4444 , piece_id=N3Z2NAQO5LHBPSABVMX4UBAAGUDDGDZCYGWX3NXWP6LUL5VJ74BA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.306609824 (4h46m37.893666035s) [noise=true , address=104.36.82.13:4190 , piece_id=PVB3Y7KXU47SOQCZBKW4AUZ56XMLMJQ23WPM73BKHUMZKARYOCCA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.307083188 (4h46m37.894139409s) [noise=true , address=104.36.82.46:4511 , piece_id=XTWLBKT2PX36GPDNTPPQ33HLJE2QMLNTVUMXVDEWJJO6IGC4HCXA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.333002740 (4h46m37.92005898s) [noise=true , address=104.36.82.11:4130 , piece_id=434EKDAN537AQFR342ZKLLENUIZAEK4PL2UTXDRJU7NJJ5HYHLUQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.335712287 (4h46m37.922768508s) [noise=true , address=79.127.220.97:4396 , piece_id=HD3NDYSYACLTEX2OYRUDYNK5J2DREH3DTNSJSDO745Q6NCC7AXGA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.364828819 (4h46m37.95188508s) [noise=true , address=104.36.82.54:4183 , piece_id=PIBOQ56366UKNWMH43GSS6TW66GFPMSJOB3UQ3TKMEYEYPKVVFOQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.415758456 (4h46m38.002814697s) [noise=true , address=104.36.82.37:4680 , piece_id=3LPKEUJ5HCZL4XY7YQOI2CROEHYNBFSZV6EUFDU2WV363FWQ644Q, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.427338251 (4h46m38.014394462s) [noise=true , address=104.36.82.43:4849 , piece_id=ZVVOBDRSIRLKPAOF23RMYRBXRNCNMQF3TJXYAW77ABOTGYUMXGHQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.490461549 (4h46m38.07751779s) [noise=true , address=79.127.201.215:4354 , piece_id=4VVIMMVTX6SCVPFAODIC7JS5PVULFS2SCXMKFJKA72NXKEK62K3Q, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.581523329 (4h46m38.16857954s) [noise=true , address=104.36.82.52:4108 , piece_id=AHQKVW323AG3TIIDCPX4HYFISVIY4V2476ZQM5GXLGLQD2PJ2KVA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.581552854 (4h46m38.168609065s) [seg_pos={0 0}, upload=4936]: Segment reached optimal threshold of 14 pieces. Aug 7 21:13:24.581774899 (4h46m38.16883112s) [noise=true , address=79.127.205.233:4643 , piece_id=IRDF6ZKJUCKCPSKAOCHW77H7O5ORLI43QULF2KWJ4O6SJTP373BQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cVbdibJiX3T4fEUjeVBNNTTNaMb7m3QYDmxh17Dic5wVUQ9fx): protocol: expected piece hash; context canceled; EOF Aug 7 21:13:24.581831154 (4h46m38.168887375s) [noise=true , address=104.36.82.62:4427 , piece_id=QMUVKOOD3THFKQAKBI3RQWH7JOP34MNNDOMKUXXBF4ZRVSYUJHJQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bx9fnLeXTPskzQB1Ad45qnq4CRivoGy7KXALY35dzVGF6S5Ep): protocol: write tcp 5.161.89.252:41822->104.36.82.62:4427: use of closed network connection; write tcp 5.161.89.252:41822->104.36.82.62:4427: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41822->104.36.82.62:4427: use of closed network connection Aug 7 21:13:24.581866029 (4h46m38.168922239s) [noise=true , address=64.49.13.154:4933 , piece_id=HGHDC3TAI5HTPACRHKMJMLCMMF2AZOMT4DO5LIQEIECYYKCHMIRA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n5Mn8FNLXxFWtbbaMv8VuyxuXe6PiiMVtLETzm5aKfxhEykt3): protocol: write tcp 5.161.89.252:43274->64.49.13.154:4933: use of closed network connection; write tcp 5.161.89.252:43274->64.49.13.154:4933: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:43274->64.49.13.154:4933: use of closed network connection Aug 7 21:13:24.581904450 (4h46m38.168960661s) [noise=true , address=64.49.13.27:4963 , piece_id=MN6MBMOOPEIPZRW5R4NQM6AEE3AHYX33MIXOSIIZEGHDPJEKAOPA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: write tcp 5.161.89.252:54020->64.49.13.27:4963: use of closed network connection; write tcp 5.161.89.252:54020->64.49.13.27:4963: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:54020->64.49.13.27:4963: use of closed network connection Aug 7 21:13:24.581925911 (4h46m38.168982132s) [seg_pos={0 0}, upload=4936]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:13:24.582013845 (4h46m38.169070066s) [upload=4936]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:13:24.582018403 (4h46m38.169074624s) [upload=4936]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:00.743765287 (4h48m14.330821508s) [upload=4936]: Done uploading segments... Aug 7 21:15:00.743795124 (4h48m14.330851345s) [upload=4936]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:15:00.812344761 (4h48m14.399400973s) [upload=4967]: Starting upload Aug 7 21:15:00.812384405 (4h48m14.399440606s) [upload=4967]: Done starting upload Aug 7 21:15:00.812509518 (4h48m14.399565729s) [upload=4967]: Uploading segments... Aug 7 21:15:00.812592684 (4h48m14.399648895s) [upload=4967]: Got next segment. Inline: false Aug 7 21:15:00.812634802 (4h48m14.399691023s) [upload=4967]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:00.812639241 (4h48m14.399695462s) [upload=4967]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:16:00.800009657 (4h49m14.387065878s) [upload=4967]: Next segment err: context deadline exceeded Aug 7 21:16:00.800074909 (4h49m14.38713113s) [upload=4967]: Done uploading segments... Aug 7 21:16:00.800088334 (4h49m14.387144545s) [upload=4967]: Upload finished. err: context deadline exceeded Duration: 1m0.00069055s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 980.009985ms FAIL (2025-08-07 21:15:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 980.032127ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XX.X] duration: 1m0.001458078s FAIL (2025-08-07 21:15:51 UTC) operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001491489s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000395109s FAIL (2025-08-07 21:15:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== FKEX4CBRRNCIBJCSLLSVKCTFRTYRD3U4TLLQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.734579330 (4h46m23.321635571s) [noise=true , address=45.91.116.4:4937 , piece_id=HUS3E4LN6CYOXDY2DIEK4OATMZPZADDVIQO3CM3L5QEMB567P3KA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.734600279 (4h46m23.321656521s) [noise=true , address=104.36.82.15:4259 , piece_id=AZDG4LGBEPEVUA4KZH24JWD2BCFEUA2IGBFWC57N4D5WVXBJ27PA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.734966383 (4h46m23.322022624s) [noise=true , address=77.90.31.16:4733 , piece_id=EMDHMK2PGYUT3SUXOVNEKLRHOH3XY4WNXON4OXTERRMZI43GCROA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735136770 (4h46m23.322193012s) [noise=true , address=77.90.31.19:4841 , piece_id=AKJV4UNYIXKLSI4Y5STPWBXJ3AFZERBSVCIER53F23KL5WABQFTQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735341183 (4h46m23.322397425s) [noise=true , address=104.36.82.13:4185 , piece_id=JRWMK5KTTCJIL5JJ4YVJ3GHS6BYCZP3CRWM4AN4ZL2UU4HJADBSQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735661972 (4h46m23.322718213s) [noise=true , address=104.36.82.7:4910 , piece_id=HFOIPKET4SKDQ4LRX54T7VOA7V2Y46MZ6RH5DED63ZXTTDYZTHQA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735696867 (4h46m23.322753117s) [noise=true , address=104.36.82.12:4151 , piece_id=OIWOXCCCVAW3B5ISSLNWASC6TD2WSWC4J3EJMF554OHVQMCZUIWQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735736471 (4h46m23.322792711s) [noise=true , address=77.90.31.17:4772 , piece_id=QUJ2AZMAXL2FFEWVASVIN734LGSMBFRXHX5KORJLQNGIXNN64ZVA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736212920 (4h46m23.323269172s) [noise=true , address=104.36.82.2:4729 , piece_id=KQOVEHNICVG5FWPV2B7CP3U6SMP56T7PRNZOG65QDAX2MP4SEN7Q, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736236384 (4h46m23.323292625s) [noise=true , address=77.90.31.18:4807 , piece_id=GYJGSHU2PC4DM5VA6SLHGM7S45KOLLBLBTMKD6LT5LMAEMHKGK5Q, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736629768 (4h46m23.323686009s) [noise=true , address=104.36.82.13:4187 , piece_id=J3E4KCSQZDYKXQMOTRSUUUIU2SZIRFBM7WD32FCFJL7V7DILAQXA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736782593 (4h46m23.323838835s) [noise=true , address=45.91.116.12:4225 , piece_id=EPHX3ZYTEBJUTQP56HQSL4GPW5SBMC7KDNSR6BD7CRQTMCDUZ2DQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.737971863 (4h46m23.325028114s) [noise=true , address=104.36.82.9:4006 , piece_id=EH5ZUHUO55F7YYVD2TSWXDIUX65GY2VVVMBW5DOFRADL3WO3ZQGA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.841271227 (4h46m23.428327438s) [upload=4938]: Next returned nil segment Aug 7 21:13:09.841281055 (4h46m23.428337266s) [upload=4938]: Waiting for error group managing segments... Aug 7 21:13:09.853631449 (4h46m23.440687659s) [noise=true , address=45.91.116.4:4937 , piece_id=HUS3E4LN6CYOXDY2DIEK4OATMZPZADDVIQO3CM3L5QEMB567P3KA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.854204258 (4h46m23.441260499s) [noise=true , address=45.91.116.12:4225 , piece_id=EPHX3ZYTEBJUTQP56HQSL4GPW5SBMC7KDNSR6BD7CRQTMCDUZ2DQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.854658546 (4h46m23.441714757s) [noise=true , address=45.91.116.14:4191 , piece_id=23HFDLS3OMLHX4EBBB3N6ORXPOJGZ4TSLXD622Z5QW57I7F33YSQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.855360527 (4h46m23.442416778s) [noise=true , address=77.90.31.18:4807 , piece_id=GYJGSHU2PC4DM5VA6SLHGM7S45KOLLBLBTMKD6LT5LMAEMHKGK5Q, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.855562183 (4h46m23.442618394s) [noise=true , address=104.36.82.13:4185 , piece_id=JRWMK5KTTCJIL5JJ4YVJ3GHS6BYCZP3CRWM4AN4ZL2UU4HJADBSQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.856428770 (4h46m23.443485021s) [noise=true , address=77.90.31.19:4841 , piece_id=AKJV4UNYIXKLSI4Y5STPWBXJ3AFZERBSVCIER53F23KL5WABQFTQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.856590482 (4h46m23.443646684s) [noise=true , address=104.36.82.2:4729 , piece_id=KQOVEHNICVG5FWPV2B7CP3U6SMP56T7PRNZOG65QDAX2MP4SEN7Q, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.858568354 (4h46m23.445624565s) [noise=true , address=45.91.116.13:4155 , piece_id=BXSMK5DW67MELF2KN6CIDHJ5HOSF7MZBRVSA7WQ45UV4SLI5XHWQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.858850201 (4h46m23.445906441s) [noise=true , address=77.90.31.16:4733 , piece_id=EMDHMK2PGYUT3SUXOVNEKLRHOH3XY4WNXON4OXTERRMZI43GCROA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.859375651 (4h46m23.446431862s) [noise=true , address=77.90.31.17:4772 , piece_id=QUJ2AZMAXL2FFEWVASVIN734LGSMBFRXHX5KORJLQNGIXNN64ZVA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.859727598 (4h46m23.446783819s) [noise=true , address=104.36.82.13:4187 , piece_id=J3E4KCSQZDYKXQMOTRSUUUIU2SZIRFBM7WD32FCFJL7V7DILAQXA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862181970 (4h46m23.44923821s) [noise=true , address=104.36.82.12:4151 , piece_id=OIWOXCCCVAW3B5ISSLNWASC6TD2WSWC4J3EJMF554OHVQMCZUIWQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862286064 (4h46m23.449342285s) [noise=true , address=104.36.82.7:4910 , piece_id=HFOIPKET4SKDQ4LRX54T7VOA7V2Y46MZ6RH5DED63ZXTTDYZTHQA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862295481 (4h46m23.449351692s) [seg_pos={0 0}, upload=4938]: Segment reached optimal threshold of 13 pieces. Aug 7 21:13:09.862540559 (4h46m23.44959677s) [noise=true , address=104.36.82.9:4006 , piece_id=EH5ZUHUO55F7YYVD2TSWXDIUX65GY2VVVMBW5DOFRADL3WO3ZQGA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection; write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection Aug 7 21:13:09.862594711 (4h46m23.449650932s) [noise=true , address=104.36.82.5:4839 , piece_id=JFXBI2UEF5IPMF3NFKEX4CBRRNCIBJCSLLSVKCTFRTYRD3U4TLLQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection; write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection Aug 7 21:13:09.862660843 (4h46m23.449717085s) [noise=true , address=104.36.82.15:4259 , piece_id=AZDG4LGBEPEVUA4KZH24JWD2BCFEUA2IGBFWC57N4D5WVXBJ27PA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection; write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection Aug 7 21:13:09.862727478 (4h46m23.449783718s) [seg_pos={0 0}, upload=4938]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:13:09.862892626 (4h46m23.449948867s) [upload=4938]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:13:09.862899970 (4h46m23.449956211s) [upload=4938]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:09.912447959 (4h46m23.499504169s) [upload=4938]: Done uploading segments... Aug 7 21:13:09.912480760 (4h46m23.499536981s) [upload=4938]: Upload finished. err: <nil> Aug 7 21:14:19.537010001 (4h47m33.124066212s) [upload=4958]: Starting upload Aug 7 21:14:19.537096253 (4h47m33.124152504s) [upload=4958]: Done starting upload Aug 7 21:14:19.537202491 (4h47m33.124258732s) [upload=4958]: Uploading segments... Aug 7 21:14:19.537385743 (4h47m33.124441964s) [upload=4958]: Got next segment. Inline: false Aug 7 21:14:19.537463918 (4h47m33.124520139s) [upload=4958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:19.537469729 (4h47m33.12452595s) [upload=4958]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:49.522961209 (4h48m3.11001742s) [upload=4958]: Next segment err: context deadline exceeded Aug 7 21:14:49.523117610 (4h48m3.110173821s) [upload=4958]: Done uploading segments... Aug 7 21:14:49.523153057 (4h48m3.110209278s) [upload=4958]: Upload finished. err: context deadline exceeded Aug 7 21:15:15.778236986 (4h48m29.365293237s) [upload=4971]: Starting upload Aug 7 21:15:15.778363202 (4h48m29.365419443s) [upload=4971]: Done starting upload Aug 7 21:15:15.778550181 (4h48m29.365606442s) [upload=4971]: Uploading segments... Aug 7 21:15:15.778800839 (4h48m29.36585709s) [upload=4971]: Got next segment. Inline: false Aug 7 21:15:15.778897088 (4h48m29.36595334s) [upload=4971]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:15:15.778908650 (4h48m29.365964891s) [upload=4971]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:15:45.754152630 (4h48m59.341208851s) [upload=4971]: Next segment err: context deadline exceeded Aug 7 21:15:45.754250592 (4h48m59.341306803s) [upload=4971]: Done uploading segments... Aug 7 21:15:45.754272974 (4h48m59.341329186s) [upload=4971]: Upload finished. err: context deadline exceeded Duration: 30.000414174s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.000673528s FAIL (2025-08-07 21:15:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000688876s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XX..XX] duration: 30.000501613s FAIL (2025-08-07 21:15:26 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.000533624s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.000590184s FAIL (2025-08-07 21:15:24 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== .BatchRequestItem_ObjectCommit Aug 7 21:06:40.686568224 (4h39m54.273624466s) [upload=4826]: Done uploading segments... Aug 7 21:06:40.686587321 (4h39m54.273643561s) [upload=4826]: Upload finished. err: <nil> Aug 7 21:07:44.924655027 (4h40m58.511711268s) [upload=4843]: Starting upload Aug 7 21:07:44.924737851 (4h40m58.511794093s) [upload=4843]: Done starting upload Aug 7 21:07:44.924828131 (4h40m58.511884362s) [upload=4843]: Uploading segments... Aug 7 21:07:44.924914792 (4h40m58.511971083s) [upload=4843]: Got next segment. Inline: true Aug 7 21:07:44.924922236 (4h40m58.511978477s) [upload=4843]: Waiting for error group managing segments... Aug 7 21:07:44.924972159 (4h40m58.51202839s) [upload=4843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:07:44.924977459 (4h40m58.5120337s) [upload=4843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:07:44.924981096 (4h40m58.512037326s) [upload=4843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:07:44.986648167 (4h40m58.573704407s) [upload=4843]: Done uploading segments... Aug 7 21:07:44.986675408 (4h40m58.573731649s) [upload=4843]: Upload finished. err: <nil> Aug 7 21:08:51.184262427 (4h42m4.771318639s) [upload=4861]: Starting upload Aug 7 21:08:51.184314965 (4h42m4.771371176s) [upload=4861]: Done starting upload Aug 7 21:08:51.184366822 (4h42m4.771423043s) [upload=4861]: Uploading segments... Aug 7 21:08:51.184393902 (4h42m4.771450113s) [upload=4861]: Got next segment. Inline: true Aug 7 21:08:51.184397439 (4h42m4.77145365s) [upload=4861]: Waiting for error group managing segments... Aug 7 21:08:51.184429829 (4h42m4.771486041s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:08:51.184433546 (4h42m4.771489758s) [upload=4861]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:08:51.184435881 (4h42m4.771492092s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:08:51.240315229 (4h42m4.82737147s) [upload=4861]: Done uploading segments... Aug 7 21:08:51.240335075 (4h42m4.827391316s) [upload=4861]: Upload finished. err: <nil> Aug 7 21:09:58.632781085 (4h43m12.219837326s) [upload=4880]: Starting upload Aug 7 21:09:58.632867045 (4h43m12.219923286s) [upload=4880]: Done starting upload Aug 7 21:09:58.632946213 (4h43m12.220002465s) [upload=4880]: Uploading segments... Aug 7 21:09:58.632990476 (4h43m12.220046716s) [upload=4880]: Got next segment. Inline: true Aug 7 21:09:58.632996186 (4h43m12.220052437s) [upload=4880]: Waiting for error group managing segments... Aug 7 21:09:58.633044917 (4h43m12.220101159s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:09:58.633050438 (4h43m12.220106679s) [upload=4880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:09:58.633077508 (4h43m12.220133759s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:09:58.687844850 (4h43m12.274901172s) [upload=4880]: Done uploading segments... Aug 7 21:09:58.687873194 (4h43m12.274929435s) [upload=4880]: Upload finished. err: <nil> Aug 7 21:10:48.980309603 (4h44m2.567365814s) [upload=4895]: Starting upload Aug 7 21:10:48.980351321 (4h44m2.567407542s) [upload=4895]: Done starting upload Aug 7 21:10:48.980412926 (4h44m2.567469137s) [upload=4895]: Uploading segments... Aug 7 21:10:48.980468068 (4h44m2.567524309s) [upload=4895]: Got next segment. Inline: true Aug 7 21:10:48.980474410 (4h44m2.567530621s) [upload=4895]: Waiting for error group managing segments... Aug 7 21:10:48.980519745 (4h44m2.567575966s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:48.980525596 (4h44m2.567581807s) [upload=4895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:48.980529363 (4h44m2.567585584s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:49.036078074 (4h44m2.623134295s) [upload=4895]: Done uploading segments... Aug 7 21:10:49.036092611 (4h44m2.623148821s) [upload=4895]: Upload finished. err: <nil> Aug 7 21:11:10.299224768 (4h44m23.886281009s) [upload=4903]: Starting upload Aug 7 21:11:10.299304937 (4h44m23.886361178s) [upload=4903]: Done starting upload Aug 7 21:11:10.299368445 (4h44m23.886424686s) [upload=4903]: Uploading segments... Aug 7 21:11:10.299409052 (4h44m23.886465292s) [upload=4903]: Got next segment. Inline: true Aug 7 21:11:10.299414652 (4h44m23.886470893s) [upload=4903]: Waiting for error group managing segments... Aug 7 21:11:10.299460468 (4h44m23.886516709s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:10.299465988 (4h44m23.886522229s) [upload=4903]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:10.299469444 (4h44m23.886525685s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:10.444467755 (4h44m24.031523996s) [upload=4903]: Done uploading segments... Aug 7 21:11:10.444485819 (4h44m24.03154206s) [upload=4903]: Upload finished. err: <nil> Aug 7 21:12:08.543001654 (4h45m22.130057896s) [upload=4919]: Starting upload Aug 7 21:12:08.543123191 (4h45m22.130179432s) [upload=4919]: Done starting upload Aug 7 21:12:08.543216246 (4h45m22.130272487s) [upload=4919]: Uploading segments... Aug 7 21:12:08.543261810 (4h45m22.130318051s) [upload=4919]: Got next segment. Inline: true Aug 7 21:12:08.543267651 (4h45m22.130323892s) [upload=4919]: Waiting for error group managing segments... Aug 7 21:12:08.543313246 (4h45m22.130369488s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:08.543319278 (4h45m22.130375519s) [upload=4919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:08.543323075 (4h45m22.130379316s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:08.630954371 (4h45m22.218010612s) [upload=4919]: Done uploading segments... Aug 7 21:12:08.630979197 (4h45m22.218035449s) [upload=4919]: Upload finished. err: <nil> Aug 7 21:12:30.034507380 (4h45m43.62156362s) [upload=4926]: Starting upload Aug 7 21:12:30.034584302 (4h45m43.621640544s) [upload=4926]: Done starting upload Aug 7 21:12:30.034643523 (4h45m43.621699754s) [upload=4926]: Uploading segments... Aug 7 21:12:30.034681965 (4h45m43.621738206s) [upload=4926]: Got next segment. Inline: true Aug 7 21:12:30.034687645 (4h45m43.621743886s) [upload=4926]: Waiting for error group managing segments... Aug 7 21:12:30.034730766 (4h45m43.621787007s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:30.034735976 (4h45m43.621792207s) [upload=4926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:30.034739642 (4h45m43.621795883s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:00.022476261 (4h46m13.609532503s) [upload=4926]: Done uploading segments... Aug 7 21:13:00.022497211 (4h46m13.609553452s) [upload=4926]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:13:15.702096720 (4h46m29.289152941s) [upload=4939]: Starting upload Aug 7 21:13:15.702154518 (4h46m29.289210739s) [upload=4939]: Done starting upload Aug 7 21:13:15.702196716 (4h46m29.289252928s) [upload=4939]: Uploading segments... Aug 7 21:13:15.702224368 (4h46m29.289280589s) [upload=4939]: Got next segment. Inline: true Aug 7 21:13:15.702228446 (4h46m29.289284657s) [upload=4939]: Waiting for error group managing segments... Aug 7 21:13:15.702261458 (4h46m29.289317669s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:15.702265306 (4h46m29.289321527s) [upload=4939]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:15.702267880 (4h46m29.289324101s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:45.689162551 (4h46m59.276218792s) [upload=4939]: Done uploading segments... Aug 7 21:13:45.689208046 (4h46m59.276264277s) [upload=4939]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:13.522007492 (4h47m27.109063713s) [upload=4955]: Starting upload Aug 7 21:14:13.522102209 (4h47m27.10915843s) [upload=4955]: Done starting upload Aug 7 21:14:13.522154938 (4h47m27.109211179s) [upload=4955]: Uploading segments... Aug 7 21:14:13.522235488 (4h47m27.109291729s) [upload=4955]: Got next segment. Inline: true Aug 7 21:14:13.522242682 (4h47m27.109298922s) [upload=4955]: Waiting for error group managing segments... Aug 7 21:14:13.522293467 (4h47m27.109349708s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:13.522299518 (4h47m27.109355759s) [upload=4955]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:13.522303275 (4h47m27.109359516s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:43.507883903 (4h47m57.094940144s) [upload=4955]: Done uploading segments... Aug 7 21:14:43.507905914 (4h47m57.094962156s) [upload=4955]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:54.825501522 (4h48m8.412557734s) [upload=4964]: Starting upload Aug 7 21:14:54.825580730 (4h48m8.412636951s) [upload=4964]: Done starting upload Aug 7 21:14:54.825629411 (4h48m8.412685632s) [upload=4964]: Uploading segments... Aug 7 21:14:54.825660400 (4h48m8.41271661s) [upload=4964]: Got next segment. Inline: true Aug 7 21:14:54.825664547 (4h48m8.412720758s) [upload=4964]: Waiting for error group managing segments... Aug 7 21:14:54.825705873 (4h48m8.412762115s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:54.825711855 (4h48m8.412768076s) [upload=4964]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:54.825715662 (4h48m8.412771883s) [upload=4964]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:24.812961328 (4h48m38.400017569s) [upload=4964]: Done uploading segments... Aug 7 21:15:24.812988770 (4h48m38.40004501s) [upload=4964]: Upload finished. err: metaclient: context deadline exceeded Duration: 30.000613219s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000340788s FAIL (2025-08-07 21:15:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000356799s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000521213s FAIL (2025-08-07 21:15:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== em_SegmentMakeInline Aug 7 21:07:15.042305125 (4h40m28.629361366s) [upload=4833]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:07:15.126405271 (4h40m28.713461512s) [upload=4833]: Done uploading segments... Aug 7 21:07:15.126421482 (4h40m28.713477722s) [upload=4833]: Upload finished. err: <nil> Aug 7 21:07:53.306326917 (4h41m6.893383169s) [upload=4846]: Starting upload Aug 7 21:07:53.306404583 (4h41m6.893460824s) [upload=4846]: Done starting upload Aug 7 21:07:53.306480694 (4h41m6.893536946s) [upload=4846]: Uploading segments... Aug 7 21:07:53.306515840 (4h41m6.893572082s) [upload=4846]: Got next segment. Inline: true Aug 7 21:07:53.306521151 (4h41m6.893577391s) [upload=4846]: Waiting for error group managing segments... Aug 7 21:07:53.306567176 (4h41m6.893623417s) [upload=4846]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:07:53.306572816 (4h41m6.893629057s) [upload=4846]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:07:53.306576423 (4h41m6.893632675s) [upload=4846]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:07:53.372716776 (4h41m6.959773018s) [upload=4846]: Done uploading segments... Aug 7 21:07:53.372735220 (4h41m6.959791462s) [upload=4846]: Upload finished. err: <nil> Aug 7 21:08:39.614384111 (4h41m53.201440353s) [upload=4858]: Starting upload Aug 7 21:08:39.614474791 (4h41m53.201531031s) [upload=4858]: Done starting upload Aug 7 21:08:39.614542067 (4h41m53.201598308s) [upload=4858]: Uploading segments... Aug 7 21:08:39.614579557 (4h41m53.201635799s) [upload=4858]: Got next segment. Inline: true Aug 7 21:08:39.614585047 (4h41m53.201641289s) [upload=4858]: Waiting for error group managing segments... Aug 7 21:08:39.614629911 (4h41m53.201686161s) [upload=4858]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:08:39.614635952 (4h41m53.201692193s) [upload=4858]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:08:39.614639619 (4h41m53.20169586s) [upload=4858]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:08:39.738955514 (4h41m53.326011754s) [upload=4858]: Done uploading segments... Aug 7 21:08:39.738974128 (4h41m53.32603037s) [upload=4858]: Upload finished. err: <nil> Aug 7 21:09:18.312602069 (4h42m31.899658319s) [upload=4869]: Starting upload Aug 7 21:09:18.312710330 (4h42m31.899766582s) [upload=4869]: Done starting upload Aug 7 21:09:18.312805559 (4h42m31.89986181s) [upload=4869]: Uploading segments... Aug 7 21:09:18.312872103 (4h42m31.899928344s) [upload=4869]: Got next segment. Inline: true Aug 7 21:09:18.312879376 (4h42m31.899935618s) [upload=4869]: Waiting for error group managing segments... Aug 7 21:09:18.312928889 (4h42m31.89998514s) [upload=4869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:09:18.312934650 (4h42m31.8999909s) [upload=4869]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:09:18.312938246 (4h42m31.899994497s) [upload=4869]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:09:18.367751803 (4h42m31.954808044s) [upload=4869]: Done uploading segments... Aug 7 21:09:18.367774827 (4h42m31.954831078s) [upload=4869]: Upload finished. err: <nil> Aug 7 21:10:26.543295840 (4h43m40.130352072s) [upload=4887]: Starting upload Aug 7 21:10:26.543341496 (4h43m40.130397717s) [upload=4887]: Done starting upload Aug 7 21:10:26.543385698 (4h43m40.130441919s) [upload=4887]: Uploading segments... Aug 7 21:10:26.543414672 (4h43m40.130470883s) [upload=4887]: Got next segment. Inline: true Aug 7 21:10:26.543418339 (4h43m40.13047455s) [upload=4887]: Waiting for error group managing segments... Aug 7 21:10:26.543446882 (4h43m40.130503103s) [upload=4887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:26.543450790 (4h43m40.130507011s) [upload=4887]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:26.543453054 (4h43m40.130509265s) [upload=4887]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:26.611321150 (4h43m40.198377371s) [upload=4887]: Done uploading segments... Aug 7 21:10:26.611339073 (4h43m40.198395294s) [upload=4887]: Upload finished. err: <nil> Aug 7 21:10:58.699165641 (4h44m12.286221892s) [upload=4897]: Starting upload Aug 7 21:10:58.699245781 (4h44m12.286302022s) [upload=4897]: Done starting upload Aug 7 21:10:58.699314669 (4h44m12.286370911s) [upload=4897]: Uploading segments... Aug 7 21:10:58.699373410 (4h44m12.286429651s) [upload=4897]: Got next segment. Inline: true Aug 7 21:10:58.699380724 (4h44m12.286436964s) [upload=4897]: Waiting for error group managing segments... Aug 7 21:10:58.699426338 (4h44m12.286482579s) [upload=4897]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:58.699432380 (4h44m12.28648862s) [upload=4897]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:58.699436066 (4h44m12.286492317s) [upload=4897]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:58.775720472 (4h44m12.362776712s) [upload=4897]: Done uploading segments... Aug 7 21:10:58.775736111 (4h44m12.362792352s) [upload=4897]: Upload finished. err: <nil> Aug 7 21:11:31.613155564 (4h44m45.200211785s) [upload=4908]: Starting upload Aug 7 21:11:31.613232477 (4h44m45.200288698s) [upload=4908]: Done starting upload Aug 7 21:11:31.613307799 (4h44m45.20036401s) [upload=4908]: Uploading segments... Aug 7 21:11:31.613347051 (4h44m45.200403272s) [upload=4908]: Got next segment. Inline: true Aug 7 21:11:31.613352932 (4h44m45.200409153s) [upload=4908]: Waiting for error group managing segments... Aug 7 21:11:31.613404179 (4h44m45.2004604s) [upload=4908]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:31.613410591 (4h44m45.200466842s) [upload=4908]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:31.613414508 (4h44m45.200470729s) [upload=4908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:31.682826735 (4h44m45.269882977s) [upload=4908]: Done uploading segments... Aug 7 21:11:31.682853265 (4h44m45.269909476s) [upload=4908]: Upload finished. err: <nil> Aug 7 21:11:51.754258397 (4h45m5.341314618s) [upload=4915]: Starting upload Aug 7 21:11:51.754330331 (4h45m5.341386542s) [upload=4915]: Done starting upload Aug 7 21:11:51.754383972 (4h45m5.341440193s) [upload=4915]: Uploading segments... Aug 7 21:11:51.754417083 (4h45m5.341473294s) [upload=4915]: Got next segment. Inline: true Aug 7 21:11:51.754420630 (4h45m5.34147684s) [upload=4915]: Waiting for error group managing segments... Aug 7 21:11:51.754453131 (4h45m5.341509352s) [upload=4915]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:51.754457108 (4h45m5.341513329s) [upload=4915]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:51.754460725 (4h45m5.341516936s) [upload=4915]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:51.830658438 (4h45m5.417714649s) [upload=4915]: Done uploading segments... Aug 7 21:11:51.830673094 (4h45m5.417729315s) [upload=4915]: Upload finished. err: <nil> Aug 7 21:12:54.778494646 (4h46m8.365550856s) [upload=4932]: Starting upload Aug 7 21:12:54.778560719 (4h46m8.36561693s) [upload=4932]: Done starting upload Aug 7 21:12:54.778608638 (4h46m8.365664859s) [upload=4932]: Uploading segments... Aug 7 21:12:54.778637322 (4h46m8.365693543s) [upload=4932]: Got next segment. Inline: true Aug 7 21:12:54.778640999 (4h46m8.36569722s) [upload=4932]: Waiting for error group managing segments... Aug 7 21:12:54.778677527 (4h46m8.365733739s) [upload=4932]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:54.778681655 (4h46m8.365737876s) [upload=4932]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:54.778684771 (4h46m8.365740983s) [upload=4932]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:54.868239378 (4h46m8.455295599s) [upload=4932]: Done uploading segments... Aug 7 21:12:54.868253865 (4h46m8.455310086s) [upload=4932]: Upload finished. err: <nil> Aug 7 21:13:23.238906839 (4h46m36.82596306s) [upload=4942]: Starting upload Aug 7 21:13:23.238959356 (4h46m36.826015567s) [upload=4942]: Done starting upload Aug 7 21:13:23.239011013 (4h46m36.826067234s) [upload=4942]: Uploading segments... Aug 7 21:13:23.239039727 (4h46m36.826095937s) [upload=4942]: Got next segment. Inline: true Aug 7 21:13:23.239043483 (4h46m36.826099695s) [upload=4942]: Waiting for error group managing segments... Aug 7 21:13:23.239098526 (4h46m36.826154737s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:23.239104707 (4h46m36.826160919s) [upload=4942]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:23.239108474 (4h46m36.826164696s) [upload=4942]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:23.309357825 (4h46m36.896414127s) [upload=4942]: Done uploading segments... Aug 7 21:13:23.309383974 (4h46m36.896440216s) [upload=4942]: Upload finished. err: <nil> Aug 7 21:14:18.294119361 (4h47m31.881175572s) [upload=4957]: Starting upload Aug 7 21:14:18.294187468 (4h47m31.881243679s) [upload=4957]: Done starting upload Aug 7 21:14:18.294261516 (4h47m31.881317737s) [upload=4957]: Uploading segments... Aug 7 21:14:18.294305068 (4h47m31.881361289s) [upload=4957]: Got next segment. Inline: true Aug 7 21:14:18.294311409 (4h47m31.881367621s) [upload=4957]: Waiting for error group managing segments... Aug 7 21:14:18.294349491 (4h47m31.881405712s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:18.294354059 (4h47m31.88141028s) [upload=4957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:18.294356643 (4h47m31.881412854s) [upload=4957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:18.280461255 (4h48m31.867517496s) [upload=4957]: Done uploading segments... Aug 7 21:15:18.280484308 (4h48m31.867540559s) [upload=4957]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1m0.000545438s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.055323065s FAIL (2025-08-07 21:15:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.055346398s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002741051s FAIL (2025-08-07 21:15:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002758162s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m52.880268475s FAIL (2025-08-07 21:15:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 52533871s) [seg_pos={0 0}, upload=4936]: Begin upload segment... Aug 7 21:13:24.165512515 (4h46m37.752568726s) [seg_pos={0 0}, upload=4936]: Done begin upload segment. Aug 7 21:13:24.165534726 (4h46m37.752590947s) [noise=true , address=104.36.82.36:4652 , piece_id=4S6GW6S5QMKXREUOZ4K7S77A5FIA5OGIS6MIPH3X5WFAIUYUTKBQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.165556597 (4h46m37.752612838s) [noise=true , address=64.49.13.154:4933 , piece_id=HGHDC3TAI5HTPACRHKMJMLCMMF2AZOMT4DO5LIQEIECYYKCHMIRA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.165834536 (4h46m37.752890748s) [noise=true , address=104.36.82.46:4511 , piece_id=XTWLBKT2PX36GPDNTPPQ33HLJE2QMLNTVUMXVDEWJJO6IGC4HCXA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.166100532 (4h46m37.753156743s) [noise=true , address=104.36.82.43:4849 , piece_id=ZVVOBDRSIRLKPAOF23RMYRBXRNCNMQF3TJXYAW77ABOTGYUMXGHQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.166323508 (4h46m37.753379719s) [noise=true , address=104.36.82.62:4427 , piece_id=QMUVKOOD3THFKQAKBI3RQWH7JOP34MNNDOMKUXXBF4ZRVSYUJHJQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.166596959 (4h46m37.75365318s) [noise=true , address=104.36.82.20:4444 , piece_id=N3Z2NAQO5LHBPSABVMX4UBAAGUDDGDZCYGWX3NXWP6LUL5VJ74BA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.166809967 (4h46m37.753866178s) [noise=true , address=79.127.205.233:4643 , piece_id=IRDF6ZKJUCKCPSKAOCHW77H7O5ORLI43QULF2KWJ4O6SJTP373BQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.167018215 (4h46m37.754074437s) [noise=true , address=104.36.82.11:4130 , piece_id=434EKDAN537AQFR342ZKLLENUIZAEK4PL2UTXDRJU7NJJ5HYHLUQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.167316922 (4h46m37.754373133s) [noise=true , address=104.36.82.23:4457 , piece_id=3LDUSHS23Z6ZBTDHVGVNUZRFA3H2XDC6B243ISKKCYJJG3PCRUBA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.167677125 (4h46m37.754733346s) [noise=true , address=104.36.82.37:4680 , piece_id=3LPKEUJ5HCZL4XY7YQOI2CROEHYNBFSZV6EUFDU2WV363FWQ644Q, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.167886165 (4h46m37.754942376s) [noise=true , address=64.49.13.27:4963 , piece_id=MN6MBMOOPEIPZRW5R4NQM6AEE3AHYX33MIXOSIIZEGHDPJEKAOPA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.168145950 (4h46m37.755202161s) [noise=true , address=79.127.201.215:4354 , piece_id=4VVIMMVTX6SCVPFAODIC7JS5PVULFS2SCXMKFJKA72NXKEK62K3Q, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.168436171 (4h46m37.755492392s) [noise=true , address=104.36.82.13:4190 , piece_id=PVB3Y7KXU47SOQCZBKW4AUZ56XMLMJQ23WPM73BKHUMZKARYOCCA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.168641956 (4h46m37.755698177s) [noise=true , address=45.91.116.10:4113 , piece_id=NJYFA7PI7F62J2CSEDJUTIWHY2LW3KBSLK76TKOVJNXC3RDV35KA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.168845667 (4h46m37.755901877s) [noise=true , address=79.127.220.97:4396 , piece_id=HD3NDYSYACLTEX2OYRUDYNK5J2DREH3DTNSJSDO745Q6NCC7AXGA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.169076598 (4h46m37.756132819s) [noise=true , address=104.36.82.44:4872 , piece_id=LM4MNB6MDHTN5MVJNGJOWX5Z5AKGICGK5JCLHBQBM5IR5PD3P2BQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.169509506 (4h46m37.756565727s) [noise=true , address=104.36.82.54:4183 , piece_id=PIBOQ56366UKNWMH43GSS6TW66GFPMSJOB3UQ3TKMEYEYPKVVFOQ, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.169710120 (4h46m37.756766341s) [noise=true , address=104.36.82.52:4108 , piece_id=AHQKVW323AG3TIIDCPX4HYFISVIY4V2476ZQM5GXLGLQD2PJ2KVA, seg_pos={0 0}, upload=4936]: Uploading piece... Aug 7 21:13:24.260747004 (4h46m37.847803255s) [upload=4936]: Next returned nil segment Aug 7 21:13:24.260784884 (4h46m37.847841125s) [upload=4936]: Waiting for error group managing segments... Aug 7 21:13:24.282697478 (4h46m37.869753699s) [noise=true , address=104.36.82.36:4652 , piece_id=4S6GW6S5QMKXREUOZ4K7S77A5FIA5OGIS6MIPH3X5WFAIUYUTKBQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.286011234 (4h46m37.873067475s) [noise=true , address=45.91.116.10:4113 , piece_id=NJYFA7PI7F62J2CSEDJUTIWHY2LW3KBSLK76TKOVJNXC3RDV35KA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.288604064 (4h46m37.875660275s) [noise=true , address=104.36.82.23:4457 , piece_id=3LDUSHS23Z6ZBTDHVGVNUZRFA3H2XDC6B243ISKKCYJJG3PCRUBA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.289097404 (4h46m37.876153645s) [noise=true , address=104.36.82.44:4872 , piece_id=LM4MNB6MDHTN5MVJNGJOWX5Z5AKGICGK5JCLHBQBM5IR5PD3P2BQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.303818073 (4h46m37.890874284s) [noise=true , address=104.36.82.20:4444 , piece_id=N3Z2NAQO5LHBPSABVMX4UBAAGUDDGDZCYGWX3NXWP6LUL5VJ74BA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.306609824 (4h46m37.893666035s) [noise=true , address=104.36.82.13:4190 , piece_id=PVB3Y7KXU47SOQCZBKW4AUZ56XMLMJQ23WPM73BKHUMZKARYOCCA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.307083188 (4h46m37.894139409s) [noise=true , address=104.36.82.46:4511 , piece_id=XTWLBKT2PX36GPDNTPPQ33HLJE2QMLNTVUMXVDEWJJO6IGC4HCXA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.333002740 (4h46m37.92005898s) [noise=true , address=104.36.82.11:4130 , piece_id=434EKDAN537AQFR342ZKLLENUIZAEK4PL2UTXDRJU7NJJ5HYHLUQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.335712287 (4h46m37.922768508s) [noise=true , address=79.127.220.97:4396 , piece_id=HD3NDYSYACLTEX2OYRUDYNK5J2DREH3DTNSJSDO745Q6NCC7AXGA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.364828819 (4h46m37.95188508s) [noise=true , address=104.36.82.54:4183 , piece_id=PIBOQ56366UKNWMH43GSS6TW66GFPMSJOB3UQ3TKMEYEYPKVVFOQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.415758456 (4h46m38.002814697s) [noise=true , address=104.36.82.37:4680 , piece_id=3LPKEUJ5HCZL4XY7YQOI2CROEHYNBFSZV6EUFDU2WV363FWQ644Q, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.427338251 (4h46m38.014394462s) [noise=true , address=104.36.82.43:4849 , piece_id=ZVVOBDRSIRLKPAOF23RMYRBXRNCNMQF3TJXYAW77ABOTGYUMXGHQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.490461549 (4h46m38.07751779s) [noise=true , address=79.127.201.215:4354 , piece_id=4VVIMMVTX6SCVPFAODIC7JS5PVULFS2SCXMKFJKA72NXKEK62K3Q, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.581523329 (4h46m38.16857954s) [noise=true , address=104.36.82.52:4108 , piece_id=AHQKVW323AG3TIIDCPX4HYFISVIY4V2476ZQM5GXLGLQD2PJ2KVA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: <nil> Aug 7 21:13:24.581552854 (4h46m38.168609065s) [seg_pos={0 0}, upload=4936]: Segment reached optimal threshold of 14 pieces. Aug 7 21:13:24.581774899 (4h46m38.16883112s) [noise=true , address=79.127.205.233:4643 , piece_id=IRDF6ZKJUCKCPSKAOCHW77H7O5ORLI43QULF2KWJ4O6SJTP373BQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cVbdibJiX3T4fEUjeVBNNTTNaMb7m3QYDmxh17Dic5wVUQ9fx): protocol: expected piece hash; context canceled; EOF Aug 7 21:13:24.581831154 (4h46m38.168887375s) [noise=true , address=104.36.82.62:4427 , piece_id=QMUVKOOD3THFKQAKBI3RQWH7JOP34MNNDOMKUXXBF4ZRVSYUJHJQ, seg_pos={0 0}, upload=4936]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12bx9fnLeXTPskzQB1Ad45qnq4CRivoGy7KXALY35dzVGF6S5Ep): protocol: write tcp 5.161.89.252:41822->104.36.82.62:4427: use of closed network connection; write tcp 5.161.89.252:41822->104.36.82.62:4427: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:41822->104.36.82.62:4427: use of closed network connection Aug 7 21:13:24.581866029 (4h46m38.168922239s) [noise=true , address=64.49.13.154:4933 , piece_id=HGHDC3TAI5HTPACRHKMJMLCMMF2AZOMT4DO5LIQEIECYYKCHMIRA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n5Mn8FNLXxFWtbbaMv8VuyxuXe6PiiMVtLETzm5aKfxhEykt3): protocol: write tcp 5.161.89.252:43274->64.49.13.154:4933: use of closed network connection; write tcp 5.161.89.252:43274->64.49.13.154:4933: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:43274->64.49.13.154:4933: use of closed network connection Aug 7 21:13:24.581904450 (4h46m38.168960661s) [noise=true , address=64.49.13.27:4963 , piece_id=MN6MBMOOPEIPZRW5R4NQM6AEE3AHYX33MIXOSIIZEGHDPJEKAOPA, seg_pos={0 0}, upload=4936]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: write tcp 5.161.89.252:54020->64.49.13.27:4963: use of closed network connection; write tcp 5.161.89.252:54020->64.49.13.27:4963: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:54020->64.49.13.27:4963: use of closed network connection Aug 7 21:13:24.581925911 (4h46m38.168982132s) [seg_pos={0 0}, upload=4936]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:13:24.582013845 (4h46m38.169070066s) [upload=4936]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:13:24.582018403 (4h46m38.169074624s) [upload=4936]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:15:00.743765287 (4h48m14.330821508s) [upload=4936]: Done uploading segments... Aug 7 21:15:00.743795124 (4h48m14.330851345s) [upload=4936]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m52.880288803s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001182259s FAIL (2025-08-07 21:14:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== tchRequestItem_ObjectBegin Aug 7 21:13:09.660957584 (4h46m23.248013836s) [upload=4938]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:13:09.733645808 (4h46m23.320702049s) [seg_pos={0 0}, upload=4938]: Begin upload segment... Aug 7 21:13:09.733687906 (4h46m23.320744148s) [seg_pos={0 0}, upload=4938]: Done begin upload segment. Aug 7 21:13:09.733718945 (4h46m23.320775185s) [noise=true , address=45.91.116.13:4155 , piece_id=BXSMK5DW67MELF2KN6CIDHJ5HOSF7MZBRVSA7WQ45UV4SLI5XHWQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.733784085 (4h46m23.320840327s) [noise=true , address=45.91.116.14:4191 , piece_id=23HFDLS3OMLHX4EBBB3N6ORXPOJGZ4TSLXD622Z5QW57I7F33YSQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.734212937 (4h46m23.321269178s) [noise=true , address=104.36.82.5:4839 , piece_id=JFXBI2UEF5IPMF3NFKEX4CBRRNCIBJCSLLSVKCTFRTYRD3U4TLLQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.734579330 (4h46m23.321635571s) [noise=true , address=45.91.116.4:4937 , piece_id=HUS3E4LN6CYOXDY2DIEK4OATMZPZADDVIQO3CM3L5QEMB567P3KA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.734600279 (4h46m23.321656521s) [noise=true , address=104.36.82.15:4259 , piece_id=AZDG4LGBEPEVUA4KZH24JWD2BCFEUA2IGBFWC57N4D5WVXBJ27PA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.734966383 (4h46m23.322022624s) [noise=true , address=77.90.31.16:4733 , piece_id=EMDHMK2PGYUT3SUXOVNEKLRHOH3XY4WNXON4OXTERRMZI43GCROA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735136770 (4h46m23.322193012s) [noise=true , address=77.90.31.19:4841 , piece_id=AKJV4UNYIXKLSI4Y5STPWBXJ3AFZERBSVCIER53F23KL5WABQFTQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735341183 (4h46m23.322397425s) [noise=true , address=104.36.82.13:4185 , piece_id=JRWMK5KTTCJIL5JJ4YVJ3GHS6BYCZP3CRWM4AN4ZL2UU4HJADBSQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735661972 (4h46m23.322718213s) [noise=true , address=104.36.82.7:4910 , piece_id=HFOIPKET4SKDQ4LRX54T7VOA7V2Y46MZ6RH5DED63ZXTTDYZTHQA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735696867 (4h46m23.322753117s) [noise=true , address=104.36.82.12:4151 , piece_id=OIWOXCCCVAW3B5ISSLNWASC6TD2WSWC4J3EJMF554OHVQMCZUIWQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.735736471 (4h46m23.322792711s) [noise=true , address=77.90.31.17:4772 , piece_id=QUJ2AZMAXL2FFEWVASVIN734LGSMBFRXHX5KORJLQNGIXNN64ZVA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736212920 (4h46m23.323269172s) [noise=true , address=104.36.82.2:4729 , piece_id=KQOVEHNICVG5FWPV2B7CP3U6SMP56T7PRNZOG65QDAX2MP4SEN7Q, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736236384 (4h46m23.323292625s) [noise=true , address=77.90.31.18:4807 , piece_id=GYJGSHU2PC4DM5VA6SLHGM7S45KOLLBLBTMKD6LT5LMAEMHKGK5Q, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736629768 (4h46m23.323686009s) [noise=true , address=104.36.82.13:4187 , piece_id=J3E4KCSQZDYKXQMOTRSUUUIU2SZIRFBM7WD32FCFJL7V7DILAQXA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.736782593 (4h46m23.323838835s) [noise=true , address=45.91.116.12:4225 , piece_id=EPHX3ZYTEBJUTQP56HQSL4GPW5SBMC7KDNSR6BD7CRQTMCDUZ2DQ, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.737971863 (4h46m23.325028114s) [noise=true , address=104.36.82.9:4006 , piece_id=EH5ZUHUO55F7YYVD2TSWXDIUX65GY2VVVMBW5DOFRADL3WO3ZQGA, seg_pos={0 0}, upload=4938]: Uploading piece... Aug 7 21:13:09.841271227 (4h46m23.428327438s) [upload=4938]: Next returned nil segment Aug 7 21:13:09.841281055 (4h46m23.428337266s) [upload=4938]: Waiting for error group managing segments... Aug 7 21:13:09.853631449 (4h46m23.440687659s) [noise=true , address=45.91.116.4:4937 , piece_id=HUS3E4LN6CYOXDY2DIEK4OATMZPZADDVIQO3CM3L5QEMB567P3KA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.854204258 (4h46m23.441260499s) [noise=true , address=45.91.116.12:4225 , piece_id=EPHX3ZYTEBJUTQP56HQSL4GPW5SBMC7KDNSR6BD7CRQTMCDUZ2DQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.854658546 (4h46m23.441714757s) [noise=true , address=45.91.116.14:4191 , piece_id=23HFDLS3OMLHX4EBBB3N6ORXPOJGZ4TSLXD622Z5QW57I7F33YSQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.855360527 (4h46m23.442416778s) [noise=true , address=77.90.31.18:4807 , piece_id=GYJGSHU2PC4DM5VA6SLHGM7S45KOLLBLBTMKD6LT5LMAEMHKGK5Q, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.855562183 (4h46m23.442618394s) [noise=true , address=104.36.82.13:4185 , piece_id=JRWMK5KTTCJIL5JJ4YVJ3GHS6BYCZP3CRWM4AN4ZL2UU4HJADBSQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.856428770 (4h46m23.443485021s) [noise=true , address=77.90.31.19:4841 , piece_id=AKJV4UNYIXKLSI4Y5STPWBXJ3AFZERBSVCIER53F23KL5WABQFTQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.856590482 (4h46m23.443646684s) [noise=true , address=104.36.82.2:4729 , piece_id=KQOVEHNICVG5FWPV2B7CP3U6SMP56T7PRNZOG65QDAX2MP4SEN7Q, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.858568354 (4h46m23.445624565s) [noise=true , address=45.91.116.13:4155 , piece_id=BXSMK5DW67MELF2KN6CIDHJ5HOSF7MZBRVSA7WQ45UV4SLI5XHWQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.858850201 (4h46m23.445906441s) [noise=true , address=77.90.31.16:4733 , piece_id=EMDHMK2PGYUT3SUXOVNEKLRHOH3XY4WNXON4OXTERRMZI43GCROA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.859375651 (4h46m23.446431862s) [noise=true , address=77.90.31.17:4772 , piece_id=QUJ2AZMAXL2FFEWVASVIN734LGSMBFRXHX5KORJLQNGIXNN64ZVA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.859727598 (4h46m23.446783819s) [noise=true , address=104.36.82.13:4187 , piece_id=J3E4KCSQZDYKXQMOTRSUUUIU2SZIRFBM7WD32FCFJL7V7DILAQXA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862181970 (4h46m23.44923821s) [noise=true , address=104.36.82.12:4151 , piece_id=OIWOXCCCVAW3B5ISSLNWASC6TD2WSWC4J3EJMF554OHVQMCZUIWQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862286064 (4h46m23.449342285s) [noise=true , address=104.36.82.7:4910 , piece_id=HFOIPKET4SKDQ4LRX54T7VOA7V2Y46MZ6RH5DED63ZXTTDYZTHQA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: <nil> Aug 7 21:13:09.862295481 (4h46m23.449351692s) [seg_pos={0 0}, upload=4938]: Segment reached optimal threshold of 13 pieces. Aug 7 21:13:09.862540559 (4h46m23.44959677s) [noise=true , address=104.36.82.9:4006 , piece_id=EH5ZUHUO55F7YYVD2TSWXDIUX65GY2VVVMBW5DOFRADL3WO3ZQGA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc): protocol: write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection; write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:35382->104.36.82.9:4006: use of closed network connection Aug 7 21:13:09.862594711 (4h46m23.449650932s) [noise=true , address=104.36.82.5:4839 , piece_id=JFXBI2UEF5IPMF3NFKEX4CBRRNCIBJCSLLSVKCTFRTYRD3U4TLLQ, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FcPS8yXfgdFvsxFofPrNCtFDLtebQMTaLu75335kVXeQxttuv): protocol: write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection; write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:52042->104.36.82.5:4839: use of closed network connection Aug 7 21:13:09.862660843 (4h46m23.449717085s) [noise=true , address=104.36.82.15:4259 , piece_id=AZDG4LGBEPEVUA4KZH24JWD2BCFEUA2IGBFWC57N4D5WVXBJ27PA, seg_pos={0 0}, upload=4938]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection; write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:8524->104.36.82.15:4259: use of closed network connection Aug 7 21:13:09.862727478 (4h46m23.449783718s) [seg_pos={0 0}, upload=4938]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 21:13:09.862892626 (4h46m23.449948867s) [upload=4938]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:13:09.862899970 (4h46m23.449956211s) [upload=4938]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:09.912447959 (4h46m23.499504169s) [upload=4938]: Done uploading segments... Aug 7 21:13:09.912480760 (4h46m23.499536981s) [upload=4938]: Upload finished. err: <nil> Aug 7 21:14:19.537010001 (4h47m33.124066212s) [upload=4958]: Starting upload Aug 7 21:14:19.537096253 (4h47m33.124152504s) [upload=4958]: Done starting upload Aug 7 21:14:19.537202491 (4h47m33.124258732s) [upload=4958]: Uploading segments... Aug 7 21:14:19.537385743 (4h47m33.124441964s) [upload=4958]: Got next segment. Inline: false Aug 7 21:14:19.537463918 (4h47m33.124520139s) [upload=4958]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:19.537469729 (4h47m33.12452595s) [upload=4958]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:14:49.522961209 (4h48m3.11001742s) [upload=4958]: Next segment err: context deadline exceeded Aug 7 21:14:49.523117610 (4h48m3.110173821s) [upload=4958]: Done uploading segments... Aug 7 21:14:49.523153057 (4h48m3.110209278s) [upload=4958]: Upload finished. err: context deadline exceeded Duration:* 30.001200092s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.000741217s FAIL (2025-08-07 21:14:43 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:05:38.586569701 (4h38m52.173625942s) [upload=4810]: Done uploading segments... Aug 7 21:05:38.586589348 (4h38m52.173645599s) [upload=4810]: Upload finished. err: <nil> Aug 7 21:06:40.621386244 (4h39m54.208442485s) [upload=4826]: Starting upload Aug 7 21:06:40.621468457 (4h39m54.208524698s) [upload=4826]: Done starting upload Aug 7 21:06:40.621541072 (4h39m54.208597313s) [upload=4826]: Uploading segments... Aug 7 21:06:40.621579394 (4h39m54.208635636s) [upload=4826]: Got next segment. Inline: true Aug 7 21:06:40.621585195 (4h39m54.208641436s) [upload=4826]: Waiting for error group managing segments... Aug 7 21:06:40.621627213 (4h39m54.208683464s) [upload=4826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:06:40.621633184 (4h39m54.208689425s) [upload=4826]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:06:40.621636951 (4h39m54.208693202s) [upload=4826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:06:40.686568224 (4h39m54.273624466s) [upload=4826]: Done uploading segments... Aug 7 21:06:40.686587321 (4h39m54.273643561s) [upload=4826]: Upload finished. err: <nil> Aug 7 21:07:44.924655027 (4h40m58.511711268s) [upload=4843]: Starting upload Aug 7 21:07:44.924737851 (4h40m58.511794093s) [upload=4843]: Done starting upload Aug 7 21:07:44.924828131 (4h40m58.511884362s) [upload=4843]: Uploading segments... Aug 7 21:07:44.924914792 (4h40m58.511971083s) [upload=4843]: Got next segment. Inline: true Aug 7 21:07:44.924922236 (4h40m58.511978477s) [upload=4843]: Waiting for error group managing segments... Aug 7 21:07:44.924972159 (4h40m58.51202839s) [upload=4843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:07:44.924977459 (4h40m58.5120337s) [upload=4843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:07:44.924981096 (4h40m58.512037326s) [upload=4843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:07:44.986648167 (4h40m58.573704407s) [upload=4843]: Done uploading segments... Aug 7 21:07:44.986675408 (4h40m58.573731649s) [upload=4843]: Upload finished. err: <nil> Aug 7 21:08:51.184262427 (4h42m4.771318639s) [upload=4861]: Starting upload Aug 7 21:08:51.184314965 (4h42m4.771371176s) [upload=4861]: Done starting upload Aug 7 21:08:51.184366822 (4h42m4.771423043s) [upload=4861]: Uploading segments... Aug 7 21:08:51.184393902 (4h42m4.771450113s) [upload=4861]: Got next segment. Inline: true Aug 7 21:08:51.184397439 (4h42m4.77145365s) [upload=4861]: Waiting for error group managing segments... Aug 7 21:08:51.184429829 (4h42m4.771486041s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:08:51.184433546 (4h42m4.771489758s) [upload=4861]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:08:51.184435881 (4h42m4.771492092s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:08:51.240315229 (4h42m4.82737147s) [upload=4861]: Done uploading segments... Aug 7 21:08:51.240335075 (4h42m4.827391316s) [upload=4861]: Upload finished. err: <nil> Aug 7 21:09:58.632781085 (4h43m12.219837326s) [upload=4880]: Starting upload Aug 7 21:09:58.632867045 (4h43m12.219923286s) [upload=4880]: Done starting upload Aug 7 21:09:58.632946213 (4h43m12.220002465s) [upload=4880]: Uploading segments... Aug 7 21:09:58.632990476 (4h43m12.220046716s) [upload=4880]: Got next segment. Inline: true Aug 7 21:09:58.632996186 (4h43m12.220052437s) [upload=4880]: Waiting for error group managing segments... Aug 7 21:09:58.633044917 (4h43m12.220101159s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:09:58.633050438 (4h43m12.220106679s) [upload=4880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:09:58.633077508 (4h43m12.220133759s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:09:58.687844850 (4h43m12.274901172s) [upload=4880]: Done uploading segments... Aug 7 21:09:58.687873194 (4h43m12.274929435s) [upload=4880]: Upload finished. err: <nil> Aug 7 21:10:48.980309603 (4h44m2.567365814s) [upload=4895]: Starting upload Aug 7 21:10:48.980351321 (4h44m2.567407542s) [upload=4895]: Done starting upload Aug 7 21:10:48.980412926 (4h44m2.567469137s) [upload=4895]: Uploading segments... Aug 7 21:10:48.980468068 (4h44m2.567524309s) [upload=4895]: Got next segment. Inline: true Aug 7 21:10:48.980474410 (4h44m2.567530621s) [upload=4895]: Waiting for error group managing segments... Aug 7 21:10:48.980519745 (4h44m2.567575966s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:48.980525596 (4h44m2.567581807s) [upload=4895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:48.980529363 (4h44m2.567585584s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:49.036078074 (4h44m2.623134295s) [upload=4895]: Done uploading segments... Aug 7 21:10:49.036092611 (4h44m2.623148821s) [upload=4895]: Upload finished. err: <nil> Aug 7 21:11:10.299224768 (4h44m23.886281009s) [upload=4903]: Starting upload Aug 7 21:11:10.299304937 (4h44m23.886361178s) [upload=4903]: Done starting upload Aug 7 21:11:10.299368445 (4h44m23.886424686s) [upload=4903]: Uploading segments... Aug 7 21:11:10.299409052 (4h44m23.886465292s) [upload=4903]: Got next segment. Inline: true Aug 7 21:11:10.299414652 (4h44m23.886470893s) [upload=4903]: Waiting for error group managing segments... Aug 7 21:11:10.299460468 (4h44m23.886516709s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:10.299465988 (4h44m23.886522229s) [upload=4903]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:10.299469444 (4h44m23.886525685s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:10.444467755 (4h44m24.031523996s) [upload=4903]: Done uploading segments... Aug 7 21:11:10.444485819 (4h44m24.03154206s) [upload=4903]: Upload finished. err: <nil> Aug 7 21:12:08.543001654 (4h45m22.130057896s) [upload=4919]: Starting upload Aug 7 21:12:08.543123191 (4h45m22.130179432s) [upload=4919]: Done starting upload Aug 7 21:12:08.543216246 (4h45m22.130272487s) [upload=4919]: Uploading segments... Aug 7 21:12:08.543261810 (4h45m22.130318051s) [upload=4919]: Got next segment. Inline: true Aug 7 21:12:08.543267651 (4h45m22.130323892s) [upload=4919]: Waiting for error group managing segments... Aug 7 21:12:08.543313246 (4h45m22.130369488s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:08.543319278 (4h45m22.130375519s) [upload=4919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:08.543323075 (4h45m22.130379316s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:08.630954371 (4h45m22.218010612s) [upload=4919]: Done uploading segments... Aug 7 21:12:08.630979197 (4h45m22.218035449s) [upload=4919]: Upload finished. err: <nil> Aug 7 21:12:30.034507380 (4h45m43.62156362s) [upload=4926]: Starting upload Aug 7 21:12:30.034584302 (4h45m43.621640544s) [upload=4926]: Done starting upload Aug 7 21:12:30.034643523 (4h45m43.621699754s) [upload=4926]: Uploading segments... Aug 7 21:12:30.034681965 (4h45m43.621738206s) [upload=4926]: Got next segment. Inline: true Aug 7 21:12:30.034687645 (4h45m43.621743886s) [upload=4926]: Waiting for error group managing segments... Aug 7 21:12:30.034730766 (4h45m43.621787007s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:30.034735976 (4h45m43.621792207s) [upload=4926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:30.034739642 (4h45m43.621795883s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:00.022476261 (4h46m13.609532503s) [upload=4926]: Done uploading segments... Aug 7 21:13:00.022497211 (4h46m13.609553452s) [upload=4926]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:13:15.702096720 (4h46m29.289152941s) [upload=4939]: Starting upload Aug 7 21:13:15.702154518 (4h46m29.289210739s) [upload=4939]: Done starting upload Aug 7 21:13:15.702196716 (4h46m29.289252928s) [upload=4939]: Uploading segments... Aug 7 21:13:15.702224368 (4h46m29.289280589s) [upload=4939]: Got next segment. Inline: true Aug 7 21:13:15.702228446 (4h46m29.289284657s) [upload=4939]: Waiting for error group managing segments... Aug 7 21:13:15.702261458 (4h46m29.289317669s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:15.702265306 (4h46m29.289321527s) [upload=4939]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:15.702267880 (4h46m29.289324101s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:45.689162551 (4h46m59.276218792s) [upload=4939]: Done uploading segments... Aug 7 21:13:45.689208046 (4h46m59.276264277s) [upload=4939]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:14:13.522007492 (4h47m27.109063713s) [upload=4955]: Starting upload Aug 7 21:14:13.522102209 (4h47m27.10915843s) [upload=4955]: Done starting upload Aug 7 21:14:13.522154938 (4h47m27.109211179s) [upload=4955]: Uploading segments... Aug 7 21:14:13.522235488 (4h47m27.109291729s) [upload=4955]: Got next segment. Inline: true Aug 7 21:14:13.522242682 (4h47m27.109298922s) [upload=4955]: Waiting for error group managing segments... Aug 7 21:14:13.522293467 (4h47m27.109349708s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:14:13.522299518 (4h47m27.109355759s) [upload=4955]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:14:13.522303275 (4h47m27.109359516s) [upload=4955]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:14:43.507883903 (4h47m57.094940144s) [upload=4955]: Done uploading segments... Aug 7 21:14:43.507905914 (4h47m57.094962156s) [upload=4955]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.00075884s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................XX..X] duration: 30.001263659s FAIL (2025-08-07 21:14:38 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.00127972s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 798.973642ms FAIL (2025-08-07 21:14:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 798.995323ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 540.281908ms FAIL (2025-08-07 21:14:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 540.301414ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 1.902849081s FAIL (2025-08-07 21:14:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.902861143s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.000657267s FAIL (2025-08-07 21:14:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000668136s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.001186351s FAIL (2025-08-07 21:13:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h37m52.468942624s) [upload=4794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:04:38.947491363 (4h37m52.534547604s) [upload=4794]: Done uploading segments... Aug 7 21:04:38.947509697 (4h37m52.534565939s) [upload=4794]: Upload finished. err: <nil> Aug 7 21:05:38.524554190 (4h38m52.111610401s) [upload=4810]: Starting upload Aug 7 21:05:38.524602881 (4h38m52.111659102s) [upload=4810]: Done starting upload Aug 7 21:05:38.524644949 (4h38m52.11170117s) [upload=4810]: Uploading segments... Aug 7 21:05:38.524670207 (4h38m52.111726428s) [upload=4810]: Got next segment. Inline: true Aug 7 21:05:38.524673593 (4h38m52.111729814s) [upload=4810]: Waiting for error group managing segments... Aug 7 21:05:38.524703629 (4h38m52.11175985s) [upload=4810]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:05:38.524707736 (4h38m52.111763957s) [upload=4810]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:05:38.524710110 (4h38m52.111766331s) [upload=4810]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:05:38.586569701 (4h38m52.173625942s) [upload=4810]: Done uploading segments... Aug 7 21:05:38.586589348 (4h38m52.173645599s) [upload=4810]: Upload finished. err: <nil> Aug 7 21:06:40.621386244 (4h39m54.208442485s) [upload=4826]: Starting upload Aug 7 21:06:40.621468457 (4h39m54.208524698s) [upload=4826]: Done starting upload Aug 7 21:06:40.621541072 (4h39m54.208597313s) [upload=4826]: Uploading segments... Aug 7 21:06:40.621579394 (4h39m54.208635636s) [upload=4826]: Got next segment. Inline: true Aug 7 21:06:40.621585195 (4h39m54.208641436s) [upload=4826]: Waiting for error group managing segments... Aug 7 21:06:40.621627213 (4h39m54.208683464s) [upload=4826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:06:40.621633184 (4h39m54.208689425s) [upload=4826]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:06:40.621636951 (4h39m54.208693202s) [upload=4826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:06:40.686568224 (4h39m54.273624466s) [upload=4826]: Done uploading segments... Aug 7 21:06:40.686587321 (4h39m54.273643561s) [upload=4826]: Upload finished. err: <nil> Aug 7 21:07:44.924655027 (4h40m58.511711268s) [upload=4843]: Starting upload Aug 7 21:07:44.924737851 (4h40m58.511794093s) [upload=4843]: Done starting upload Aug 7 21:07:44.924828131 (4h40m58.511884362s) [upload=4843]: Uploading segments... Aug 7 21:07:44.924914792 (4h40m58.511971083s) [upload=4843]: Got next segment. Inline: true Aug 7 21:07:44.924922236 (4h40m58.511978477s) [upload=4843]: Waiting for error group managing segments... Aug 7 21:07:44.924972159 (4h40m58.51202839s) [upload=4843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:07:44.924977459 (4h40m58.5120337s) [upload=4843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:07:44.924981096 (4h40m58.512037326s) [upload=4843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:07:44.986648167 (4h40m58.573704407s) [upload=4843]: Done uploading segments... Aug 7 21:07:44.986675408 (4h40m58.573731649s) [upload=4843]: Upload finished. err: <nil> Aug 7 21:08:51.184262427 (4h42m4.771318639s) [upload=4861]: Starting upload Aug 7 21:08:51.184314965 (4h42m4.771371176s) [upload=4861]: Done starting upload Aug 7 21:08:51.184366822 (4h42m4.771423043s) [upload=4861]: Uploading segments... Aug 7 21:08:51.184393902 (4h42m4.771450113s) [upload=4861]: Got next segment. Inline: true Aug 7 21:08:51.184397439 (4h42m4.77145365s) [upload=4861]: Waiting for error group managing segments... Aug 7 21:08:51.184429829 (4h42m4.771486041s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:08:51.184433546 (4h42m4.771489758s) [upload=4861]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:08:51.184435881 (4h42m4.771492092s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:08:51.240315229 (4h42m4.82737147s) [upload=4861]: Done uploading segments... Aug 7 21:08:51.240335075 (4h42m4.827391316s) [upload=4861]: Upload finished. err: <nil> Aug 7 21:09:58.632781085 (4h43m12.219837326s) [upload=4880]: Starting upload Aug 7 21:09:58.632867045 (4h43m12.219923286s) [upload=4880]: Done starting upload Aug 7 21:09:58.632946213 (4h43m12.220002465s) [upload=4880]: Uploading segments... Aug 7 21:09:58.632990476 (4h43m12.220046716s) [upload=4880]: Got next segment. Inline: true Aug 7 21:09:58.632996186 (4h43m12.220052437s) [upload=4880]: Waiting for error group managing segments... Aug 7 21:09:58.633044917 (4h43m12.220101159s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:09:58.633050438 (4h43m12.220106679s) [upload=4880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:09:58.633077508 (4h43m12.220133759s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:09:58.687844850 (4h43m12.274901172s) [upload=4880]: Done uploading segments... Aug 7 21:09:58.687873194 (4h43m12.274929435s) [upload=4880]: Upload finished. err: <nil> Aug 7 21:10:48.980309603 (4h44m2.567365814s) [upload=4895]: Starting upload Aug 7 21:10:48.980351321 (4h44m2.567407542s) [upload=4895]: Done starting upload Aug 7 21:10:48.980412926 (4h44m2.567469137s) [upload=4895]: Uploading segments... Aug 7 21:10:48.980468068 (4h44m2.567524309s) [upload=4895]: Got next segment. Inline: true Aug 7 21:10:48.980474410 (4h44m2.567530621s) [upload=4895]: Waiting for error group managing segments... Aug 7 21:10:48.980519745 (4h44m2.567575966s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:48.980525596 (4h44m2.567581807s) [upload=4895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:48.980529363 (4h44m2.567585584s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:49.036078074 (4h44m2.623134295s) [upload=4895]: Done uploading segments... Aug 7 21:10:49.036092611 (4h44m2.623148821s) [upload=4895]: Upload finished. err: <nil> Aug 7 21:11:10.299224768 (4h44m23.886281009s) [upload=4903]: Starting upload Aug 7 21:11:10.299304937 (4h44m23.886361178s) [upload=4903]: Done starting upload Aug 7 21:11:10.299368445 (4h44m23.886424686s) [upload=4903]: Uploading segments... Aug 7 21:11:10.299409052 (4h44m23.886465292s) [upload=4903]: Got next segment. Inline: true Aug 7 21:11:10.299414652 (4h44m23.886470893s) [upload=4903]: Waiting for error group managing segments... Aug 7 21:11:10.299460468 (4h44m23.886516709s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:10.299465988 (4h44m23.886522229s) [upload=4903]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:10.299469444 (4h44m23.886525685s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:10.444467755 (4h44m24.031523996s) [upload=4903]: Done uploading segments... Aug 7 21:11:10.444485819 (4h44m24.03154206s) [upload=4903]: Upload finished. err: <nil> Aug 7 21:12:08.543001654 (4h45m22.130057896s) [upload=4919]: Starting upload Aug 7 21:12:08.543123191 (4h45m22.130179432s) [upload=4919]: Done starting upload Aug 7 21:12:08.543216246 (4h45m22.130272487s) [upload=4919]: Uploading segments... Aug 7 21:12:08.543261810 (4h45m22.130318051s) [upload=4919]: Got next segment. Inline: true Aug 7 21:12:08.543267651 (4h45m22.130323892s) [upload=4919]: Waiting for error group managing segments... Aug 7 21:12:08.543313246 (4h45m22.130369488s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:08.543319278 (4h45m22.130375519s) [upload=4919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:08.543323075 (4h45m22.130379316s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:08.630954371 (4h45m22.218010612s) [upload=4919]: Done uploading segments... Aug 7 21:12:08.630979197 (4h45m22.218035449s) [upload=4919]: Upload finished. err: <nil> Aug 7 21:12:30.034507380 (4h45m43.62156362s) [upload=4926]: Starting upload Aug 7 21:12:30.034584302 (4h45m43.621640544s) [upload=4926]: Done starting upload Aug 7 21:12:30.034643523 (4h45m43.621699754s) [upload=4926]: Uploading segments... Aug 7 21:12:30.034681965 (4h45m43.621738206s) [upload=4926]: Got next segment. Inline: true Aug 7 21:12:30.034687645 (4h45m43.621743886s) [upload=4926]: Waiting for error group managing segments... Aug 7 21:12:30.034730766 (4h45m43.621787007s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:30.034735976 (4h45m43.621792207s) [upload=4926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:30.034739642 (4h45m43.621795883s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:00.022476261 (4h46m13.609532503s) [upload=4926]: Done uploading segments... Aug 7 21:13:00.022497211 (4h46m13.609553452s) [upload=4926]: Upload finished. err: metaclient: context deadline exceeded Aug 7 21:13:15.702096720 (4h46m29.289152941s) [upload=4939]: Starting upload Aug 7 21:13:15.702154518 (4h46m29.289210739s) [upload=4939]: Done starting upload Aug 7 21:13:15.702196716 (4h46m29.289252928s) [upload=4939]: Uploading segments... Aug 7 21:13:15.702224368 (4h46m29.289280589s) [upload=4939]: Got next segment. Inline: true Aug 7 21:13:15.702228446 (4h46m29.289284657s) [upload=4939]: Waiting for error group managing segments... Aug 7 21:13:15.702261458 (4h46m29.289317669s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:13:15.702265306 (4h46m29.289321527s) [upload=4939]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:13:15.702267880 (4h46m29.289324101s) [upload=4939]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:45.689162551 (4h46m59.276218792s) [upload=4939]: Done uploading segments... Aug 7 21:13:45.689208046 (4h46m59.276264277s) [upload=4939]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.001205836s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 278.260764ms FAIL (2025-08-07 21:13:40 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 278.280602ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 548.469025ms FAIL (2025-08-07 21:13:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 548.489895ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.001804563s FAIL (2025-08-07 21:13:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001822697s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001264379s FAIL (2025-08-07 21:13:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== line Aug 7 21:03:33.091375446 (4h36m46.678431677s) [upload=4778]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:03:33.155030919 (4h36m46.742087159s) [upload=4778]: Done uploading segments... Aug 7 21:03:33.155045556 (4h36m46.742101807s) [upload=4778]: Upload finished. err: <nil> Aug 7 21:04:38.881607273 (4h37m52.468663524s) [upload=4794]: Starting upload Aug 7 21:04:38.881702269 (4h37m52.468758511s) [upload=4794]: Done starting upload Aug 7 21:04:38.881780275 (4h37m52.468836516s) [upload=4794]: Uploading segments... Aug 7 21:04:38.881824989 (4h37m52.46888123s) [upload=4794]: Got next segment. Inline: true Aug 7 21:04:38.881830489 (4h37m52.468886741s) [upload=4794]: Waiting for error group managing segments... Aug 7 21:04:38.881877266 (4h37m52.468933507s) [upload=4794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:04:38.881882777 (4h37m52.468939017s) [upload=4794]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:04:38.881886373 (4h37m52.468942624s) [upload=4794]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:04:38.947491363 (4h37m52.534547604s) [upload=4794]: Done uploading segments... Aug 7 21:04:38.947509697 (4h37m52.534565939s) [upload=4794]: Upload finished. err: <nil> Aug 7 21:05:38.524554190 (4h38m52.111610401s) [upload=4810]: Starting upload Aug 7 21:05:38.524602881 (4h38m52.111659102s) [upload=4810]: Done starting upload Aug 7 21:05:38.524644949 (4h38m52.11170117s) [upload=4810]: Uploading segments... Aug 7 21:05:38.524670207 (4h38m52.111726428s) [upload=4810]: Got next segment. Inline: true Aug 7 21:05:38.524673593 (4h38m52.111729814s) [upload=4810]: Waiting for error group managing segments... Aug 7 21:05:38.524703629 (4h38m52.11175985s) [upload=4810]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:05:38.524707736 (4h38m52.111763957s) [upload=4810]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:05:38.524710110 (4h38m52.111766331s) [upload=4810]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:05:38.586569701 (4h38m52.173625942s) [upload=4810]: Done uploading segments... Aug 7 21:05:38.586589348 (4h38m52.173645599s) [upload=4810]: Upload finished. err: <nil> Aug 7 21:06:40.621386244 (4h39m54.208442485s) [upload=4826]: Starting upload Aug 7 21:06:40.621468457 (4h39m54.208524698s) [upload=4826]: Done starting upload Aug 7 21:06:40.621541072 (4h39m54.208597313s) [upload=4826]: Uploading segments... Aug 7 21:06:40.621579394 (4h39m54.208635636s) [upload=4826]: Got next segment. Inline: true Aug 7 21:06:40.621585195 (4h39m54.208641436s) [upload=4826]: Waiting for error group managing segments... Aug 7 21:06:40.621627213 (4h39m54.208683464s) [upload=4826]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:06:40.621633184 (4h39m54.208689425s) [upload=4826]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:06:40.621636951 (4h39m54.208693202s) [upload=4826]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:06:40.686568224 (4h39m54.273624466s) [upload=4826]: Done uploading segments... Aug 7 21:06:40.686587321 (4h39m54.273643561s) [upload=4826]: Upload finished. err: <nil> Aug 7 21:07:44.924655027 (4h40m58.511711268s) [upload=4843]: Starting upload Aug 7 21:07:44.924737851 (4h40m58.511794093s) [upload=4843]: Done starting upload Aug 7 21:07:44.924828131 (4h40m58.511884362s) [upload=4843]: Uploading segments... Aug 7 21:07:44.924914792 (4h40m58.511971083s) [upload=4843]: Got next segment. Inline: true Aug 7 21:07:44.924922236 (4h40m58.511978477s) [upload=4843]: Waiting for error group managing segments... Aug 7 21:07:44.924972159 (4h40m58.51202839s) [upload=4843]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:07:44.924977459 (4h40m58.5120337s) [upload=4843]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:07:44.924981096 (4h40m58.512037326s) [upload=4843]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:07:44.986648167 (4h40m58.573704407s) [upload=4843]: Done uploading segments... Aug 7 21:07:44.986675408 (4h40m58.573731649s) [upload=4843]: Upload finished. err: <nil> Aug 7 21:08:51.184262427 (4h42m4.771318639s) [upload=4861]: Starting upload Aug 7 21:08:51.184314965 (4h42m4.771371176s) [upload=4861]: Done starting upload Aug 7 21:08:51.184366822 (4h42m4.771423043s) [upload=4861]: Uploading segments... Aug 7 21:08:51.184393902 (4h42m4.771450113s) [upload=4861]: Got next segment. Inline: true Aug 7 21:08:51.184397439 (4h42m4.77145365s) [upload=4861]: Waiting for error group managing segments... Aug 7 21:08:51.184429829 (4h42m4.771486041s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:08:51.184433546 (4h42m4.771489758s) [upload=4861]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:08:51.184435881 (4h42m4.771492092s) [upload=4861]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:08:51.240315229 (4h42m4.82737147s) [upload=4861]: Done uploading segments... Aug 7 21:08:51.240335075 (4h42m4.827391316s) [upload=4861]: Upload finished. err: <nil> Aug 7 21:09:58.632781085 (4h43m12.219837326s) [upload=4880]: Starting upload Aug 7 21:09:58.632867045 (4h43m12.219923286s) [upload=4880]: Done starting upload Aug 7 21:09:58.632946213 (4h43m12.220002465s) [upload=4880]: Uploading segments... Aug 7 21:09:58.632990476 (4h43m12.220046716s) [upload=4880]: Got next segment. Inline: true Aug 7 21:09:58.632996186 (4h43m12.220052437s) [upload=4880]: Waiting for error group managing segments... Aug 7 21:09:58.633044917 (4h43m12.220101159s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:09:58.633050438 (4h43m12.220106679s) [upload=4880]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:09:58.633077508 (4h43m12.220133759s) [upload=4880]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:09:58.687844850 (4h43m12.274901172s) [upload=4880]: Done uploading segments... Aug 7 21:09:58.687873194 (4h43m12.274929435s) [upload=4880]: Upload finished. err: <nil> Aug 7 21:10:48.980309603 (4h44m2.567365814s) [upload=4895]: Starting upload Aug 7 21:10:48.980351321 (4h44m2.567407542s) [upload=4895]: Done starting upload Aug 7 21:10:48.980412926 (4h44m2.567469137s) [upload=4895]: Uploading segments... Aug 7 21:10:48.980468068 (4h44m2.567524309s) [upload=4895]: Got next segment. Inline: true Aug 7 21:10:48.980474410 (4h44m2.567530621s) [upload=4895]: Waiting for error group managing segments... Aug 7 21:10:48.980519745 (4h44m2.567575966s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:10:48.980525596 (4h44m2.567581807s) [upload=4895]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:10:48.980529363 (4h44m2.567585584s) [upload=4895]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:10:49.036078074 (4h44m2.623134295s) [upload=4895]: Done uploading segments... Aug 7 21:10:49.036092611 (4h44m2.623148821s) [upload=4895]: Upload finished. err: <nil> Aug 7 21:11:10.299224768 (4h44m23.886281009s) [upload=4903]: Starting upload Aug 7 21:11:10.299304937 (4h44m23.886361178s) [upload=4903]: Done starting upload Aug 7 21:11:10.299368445 (4h44m23.886424686s) [upload=4903]: Uploading segments... Aug 7 21:11:10.299409052 (4h44m23.886465292s) [upload=4903]: Got next segment. Inline: true Aug 7 21:11:10.299414652 (4h44m23.886470893s) [upload=4903]: Waiting for error group managing segments... Aug 7 21:11:10.299460468 (4h44m23.886516709s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:11:10.299465988 (4h44m23.886522229s) [upload=4903]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:11:10.299469444 (4h44m23.886525685s) [upload=4903]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:11:10.444467755 (4h44m24.031523996s) [upload=4903]: Done uploading segments... Aug 7 21:11:10.444485819 (4h44m24.03154206s) [upload=4903]: Upload finished. err: <nil> Aug 7 21:12:08.543001654 (4h45m22.130057896s) [upload=4919]: Starting upload Aug 7 21:12:08.543123191 (4h45m22.130179432s) [upload=4919]: Done starting upload Aug 7 21:12:08.543216246 (4h45m22.130272487s) [upload=4919]: Uploading segments... Aug 7 21:12:08.543261810 (4h45m22.130318051s) [upload=4919]: Got next segment. Inline: true Aug 7 21:12:08.543267651 (4h45m22.130323892s) [upload=4919]: Waiting for error group managing segments... Aug 7 21:12:08.543313246 (4h45m22.130369488s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:08.543319278 (4h45m22.130375519s) [upload=4919]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:08.543323075 (4h45m22.130379316s) [upload=4919]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:12:08.630954371 (4h45m22.218010612s) [upload=4919]: Done uploading segments... Aug 7 21:12:08.630979197 (4h45m22.218035449s) [upload=4919]: Upload finished. err: <nil> Aug 7 21:12:30.034507380 (4h45m43.62156362s) [upload=4926]: Starting upload Aug 7 21:12:30.034584302 (4h45m43.621640544s) [upload=4926]: Done starting upload Aug 7 21:12:30.034643523 (4h45m43.621699754s) [upload=4926]: Uploading segments... Aug 7 21:12:30.034681965 (4h45m43.621738206s) [upload=4926]: Got next segment. Inline: true Aug 7 21:12:30.034687645 (4h45m43.621743886s) [upload=4926]: Waiting for error group managing segments... Aug 7 21:12:30.034730766 (4h45m43.621787007s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:12:30.034735976 (4h45m43.621792207s) [upload=4926]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 7 21:12:30.034739642 (4h45m43.621795883s) [upload=4926]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:13:00.022476261 (4h46m13.609532503s) [upload=4926]: Done uploading segments... Aug 7 21:13:00.022497211 (4h46m13.609553452s) [upload=4926]: Upload finished. err: metaclient: context deadline exceeded Duration:* 30.001286179s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 260.117529ms FAIL (2025-08-07 21:12:45 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 260.14037ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 890.342137ms FAIL (2025-08-07 21:12:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 890.3653ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.001049074s FAIL (2025-08-07 21:12:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001067907s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 814.497486ms FAIL (2025-08-07 21:11:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 814.521009ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.001364978s FAIL (2025-08-07 21:11:38 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 30.001383663s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 717.021747ms FAIL (2025-08-07 21:11:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 717.045081ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 1.031638344s FAIL (2025-08-07 21:10:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.031681244s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 731.335647ms FAIL (2025-08-07 21:09:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 731.359179ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 714.678523ms FAIL (2025-08-07 21:08:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 714.711595ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 793.609981ms FAIL (2025-08-07 21:07:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 793.643133ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.27721523s FAIL (2025-08-07 21:07:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.277233665s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.008130774s FAIL (2025-08-07 21:06:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.008142325s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 697.562366ms FAIL (2025-08-07 21:05:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 697.587442ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 728.071662ms FAIL (2025-08-07 21:04:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 728.091721ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 786.932468ms FAIL (2025-08-07 21:03:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 786.958557ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.......................................................X...] duration: 1.02172292s FAIL (2025-08-07 21:03:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 1.021749791s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.......................................................X..] duration: 864.349026ms FAIL (2025-08-07 21:02:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 864.372379ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.......................................................X.] duration: 809.625498ms FAIL (2025-08-07 21:02:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.601346920 (4h35m2.188403141s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602014457 (4h35m2.189070678s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602175055 (4h35m2.189231277s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602655432 (4h35m2.189711644s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.602849785 (4h35m2.189906006s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603565100 (4h35m2.190621321s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603577724 (4h35m2.190633945s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603630644 (4h35m2.190686855s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.603928529 (4h35m2.19098475s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604153980 (4h35m2.191210201s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604387185 (4h35m2.191443406s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604744643 (4h35m2.191800864s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.604931691 (4h35m2.191987903s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.605523377 (4h35m2.192579598s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.609320384 (4h35m2.196376595s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Uploading piece... Aug 7 21:01:48.674305087 (4h35m2.261361299s) [upload=4741]: Next returned nil segment Aug 7 21:01:48.674322240 (4h35m2.261378451s) [upload=4741]: Waiting for error group managing segments... Aug 7 21:01:48.698882585 (4h35m2.285938796s) [noise=true , address=45.91.116.13:4184 , piece_id=CYVUHYB7MB3C5L3IA2OSA5X5LMAU4Z5I3PBKBQRTZJGY5RPEUTVA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.702260020 (4h35m2.289316221s) [noise=true , address=104.36.82.61:4375 , piece_id=EVGZEVJFKBR26BCJZVVRYQPROICZPMZUQRG6TXGKQ5MIOS2U33KQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.709310800 (4h35m2.296367012s) [noise=true , address=104.36.82.15:4273 , piece_id=E5KMED3OODLLOMH772ZLY3KPV2QKBJMEMZUKTXWZBDJVFB5PQQXQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.710480384 (4h35m2.297536605s) [noise=true , address=104.36.82.35:4582 , piece_id=7XPBU22P4VL3FOX22UCTPDSZZWR6Z2QYPEDQ5645KZPZHARKPG6Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.726552304 (4h35m2.313608525s) [noise=true , address=104.36.82.38:4711 , piece_id=EK4ORPEZSKF2LBMW5XK7FTXV5LILM4LWKN5IJDFIOXFQZ46RRJ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.735555450 (4h35m2.322611661s) [noise=true , address=104.36.82.44:4873 , piece_id=X2IB7FNHLTJ6DFRCOS4MXGOSO3NBIGPA4ZV6GNHUWPV5AAINRZ4A, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.748292316 (4h35m2.335348527s) [noise=true , address=104.36.82.60:4360 , piece_id=PBAR27QDYDYE43UYKS4EQKKVF6QZZ42UMWFBBKF7YI5HWI34GFHA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.749773620 (4h35m2.336829832s) [noise=true , address=104.36.82.13:4207 , piece_id=PTURKUKGP32DRRYU2FJCDGTYNQUPPC3HURKZX4HX52R4RKKIRD4Q, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750299954 (4h35m2.337356165s) [noise=true , address=104.36.82.23:4447 , piece_id=6ASIGUT3Z4QFCR43JOL3XLVQ5ACHTCXN7CTPOFHMLQXZ52H2A6MA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.750910052 (4h35m2.337966303s) [noise=true , address=45.91.116.7:4032 , piece_id=PYPRNETMVWPIODOI5TYMML5ULQ52ABM3UYCJJYBP7JJOUW6PNTRQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.768808221 (4h35m2.355864443s) [noise=true , address=104.36.82.4:4825 , piece_id=H7IXZDO3RMPEYP5X632YGC4IFAG6GRDAKGFXK2ULBOTKX5EPFSLA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.810369602 (4h35m2.397425823s) [noise=true , address=104.36.82.39:4753 , piece_id=2QSICT65DH3ZRGVW3JHBOPAIP737YVXFTLP6SMY2LDVQ727OQPUQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:48.840971094 (4h35m2.428027305s) [noise=true , address=104.36.82.43:4856 , piece_id=KMCGUX5ERXTCLTDIVLQ4RO7VJN2ME2PWLRNNRKUEJ2SPCD5MEGWQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007362780 (4h35m2.594418991s) [noise=true , address=104.36.82.12:4155 , piece_id=KU7AIMH6HWK2J3UUS5G2GUSHU6REMKAWSLMKZSLMK4Z4NQOBVOKQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: <nil> Aug 7 21:01:49.007381163 (4h35m2.594437374s) [seg_pos={0 0}, upload=4741]: Segment reached optimal threshold of 14 pieces. Aug 7 21:01:49.007613637 (4h35m2.594669848s) [noise=true , address=104.36.82.47:4525 , piece_id=HASUEPAY5OWMAHFKWGXYPWYWYRIR2LXSCRKYGGNFDE5KJJ5X4LFQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hRLixZtY6GptLU6MgGZAvnk4YvuDD6WbBFUePZSndN9UbpHhX): protocol: expected piece hash; context canceled; EOF Aug 7 21:01:49.007678349 (4h35m2.59473456s) [noise=true , address=193.149.164.143:4696 , piece_id=3OOJ6XGMU6Y3MTPFIFSNZYNS7SMXCTPPGVHOU7HRX5FXWL4I4TCQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1p6mcvX3TBdnYy5Eaugk1t5KPfoeJT1sxcg5TX8aSRGVhHGSsy): protocol: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:56072->193.149.164.143:4696: use of closed network connection Aug 7 21:01:49.007766854 (4h35m2.594823065s) [noise=true , address=104.36.82.58:4271 , piece_id=HC4JBQXYRY2NE3M7JCA2WYZWVMWQSMYFBHGIYRNKVD4GNAIWVKZA, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QwLHnMQHafFPYL83gE6wffvzdkvCHwzzVMLBz4KBVdE7ynuf1): protocol: read tcp 5.161.89.252:21134->104.36.82.58:4271: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 7 21:01:49.007891316 (4h35m2.594947527s) [noise=true , address=104.36.82.59:4331 , piece_id=OIXPWBMN5TUCVGF4J636JFP6DIUWDRGL7E2SM44PU3FQB44U32CQ, seg_pos={0 0}, upload=4741]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EYU9Z3iNdzeVtxWGQ8F4F7ZT6FbjaFzfr4QQpUDWUpFTrWUYU): protocol: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61270->104.36.82.59:4331: use of closed network connection Aug 7 21:01:49.007944575 (4h35m2.595000796s) [seg_pos={0 0}, upload=4741]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 7 21:01:49.008086300 (4h35m2.595142521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 21:01:49.008091300 (4h35m2.595147521s) [upload=4741]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 21:01:49.089771998 (4h35m2.676828219s) [upload=4741]: Done uploading segments... Aug 7 21:01:49.089786797 (4h35m2.676843028s) [upload=4741]: Upload finished. err: <nil> Aug 7 21:02:04.233419851 (4h35m17.820476092s) [upload=4749]: Starting upload Aug 7 21:02:04.233540527 (4h35m17.820596767s) [upload=4749]: Done starting upload Aug 7 21:02:04.233782818 (4h35m17.820839039s) [upload=4749]: Uploading segments... Aug 7 21:02:04.233984515 (4h35m17.821040766s) [upload=4749]: Got next segment. Inline: false Aug 7 21:02:04.234049276 (4h35m17.821105517s) [upload=4749]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 21:02:04.234058734 (4h35m17.821114974s) [upload=4749]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 21:02:04.237083389 (4h35m17.824139641s) [upload=4749]: Next segment err: metaclient: Too Many Requests Aug 7 21:02:04.237137491 (4h35m17.824193731s) [upload=4749]: Done uploading segments... Aug 7 21:02:04.237155915 (4h35m17.824212156s) [upload=4749]: Upload finished. err: metaclient: Too Many Requests Duration: 809.655855ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)