Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X...X.............................................] duration: 190.671569ms FAIL (2025-08-16 15:51:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 190.726092ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................XX.........X.........X.....................] duration: 17.445863452s FAIL (2025-08-16 15:51:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 17.445883269s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................XX.X......X...............] duration: 3.02291933s FAIL (2025-08-16 15:51:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 3.022943316s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X...X............................................] duration: 411.151856ms FAIL (2025-08-16 15:51:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 411.172215ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................XX.X......X..............] duration: 4.649713603s FAIL (2025-08-16 15:51:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 4.649736066s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X...X...........................................] duration: 519.787452ms FAIL (2025-08-16 15:50:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 519.815824ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.................XX.........X.........X....................] duration: 14.431683252s FAIL (2025-08-16 15:50:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 14.431707217s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................XX.X......X.............] duration: 633.541095ms FAIL (2025-08-16 15:50:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 633.563157ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X...X..........................................] duration: 301.670107ms FAIL (2025-08-16 15:50:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 301.690956ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XX.X......X............] duration: 1.291658742s FAIL (2025-08-16 15:50:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.291685762s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.................XX.........X.........X...................] duration: 21.887685484s FAIL (2025-08-16 15:49:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 21.887707475s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................XX.X......X...........] duration: 1.376410741s FAIL (2025-08-16 15:49:26 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.376433715s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X...X.........................................] duration: 233.102669ms FAIL (2025-08-16 15:49:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 233.122515ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................XX.X......X..........] duration: 1.199728066s FAIL (2025-08-16 15:48:57 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.199750569s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.................XX.........X.........X..................] duration: 24.198520074s FAIL (2025-08-16 15:48:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 24.198546794s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................XX.X......X.........] duration: 16.936385528s FAIL (2025-08-16 15:48:41 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 16.93640729s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X...X........................................] duration: 387.831682ms FAIL (2025-08-16 15:48:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 387.852941ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................XX.X......X........] duration: 5.10044542s FAIL (2025-08-16 15:47:59 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 5.100496154s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X.................XX.........X.........X.................] duration: 28.092504008s FAIL (2025-08-16 15:47:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 28.092522294s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X...X.......................................] duration: 682.632028ms FAIL (2025-08-16 15:47:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 682.671251ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................XX.X......X.......] duration: 9.710137698s FAIL (2025-08-16 15:46:55 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 9.71015925s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X.................XX.........X.........X................] duration: 15.524792442s FAIL (2025-08-16 15:46:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 15.524813951s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X.................XX.........X.........X...............] duration: 29.664092622s FAIL (2025-08-16 15:46:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 29.664118119s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................X...X......................................] duration: 297.496788ms FAIL (2025-08-16 15:46:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 297.520071ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X.................XX.........X.........X..............] duration: 35.034354245s FAIL (2025-08-16 15:45:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 35.034378331s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................XX.X......X......] duration: 1.70681941s FAIL (2025-08-16 15:45:46 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.706838374s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................................XX.X......X.....] duration: 8.498563879s FAIL (2025-08-16 15:45:21 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 8.498586572s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................X...X.....................................] duration: 854.464151ms FAIL (2025-08-16 15:45:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 854.485921ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................XX.X......X....] duration: 785.683939ms FAIL (2025-08-16 15:44:51 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 785.714536ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................X...X....................................] duration: 233.117086ms FAIL (2025-08-16 15:44:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 233.136782ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X.................XX.........X.........X.............] duration: 26.887787142s FAIL (2025-08-16 15:44:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 26.887804986s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................X...X...................................] duration: 396.631665ms FAIL (2025-08-16 15:44:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 396.659807ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X.................XX.........X.........X............] duration: 38.80926874s FAIL (2025-08-16 15:44:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 38.809281223s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................XX.X......X...] duration: 1.018579608s FAIL (2025-08-16 15:44:04 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.018601439s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................X...X..................................] duration: 144.556448ms FAIL (2025-08-16 15:43:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 144.582967ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........X.................XX.........X.........X...........] duration: 38.492225401s FAIL (2025-08-16 15:43:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 38.4922465s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................X...X.................................] duration: 132.155675ms FAIL (2025-08-16 15:43:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 132.176274ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XX.X......X..] duration: 1.058413805s FAIL (2025-08-16 15:42:55 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 1.058431807s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................X...X................................] duration: 290.712932ms FAIL (2025-08-16 15:42:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 290.735174ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X.................XX.........X.........X..........] duration: 17.593299631s FAIL (2025-08-16 15:42:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 17.593320781s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................XX.X......X.] duration: 25.953812807s FAIL (2025-08-16 15:42:16 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 25.953833235s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X...X...............................] duration: 512.138467ms FAIL (2025-08-16 15:42:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 512.165978ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X...X..............................] duration: 155.627018ms FAIL (2025-08-16 15:41:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 155.645374ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X...X.............................] duration: 920.745907ms FAIL (2025-08-16 15:41:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 920.771815ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XX.X......X] duration: 30.001768763s FAIL (2025-08-16 15:41:42 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Aug 16 15:41:12.402104593 (190h14m53.133372223s) [upload=197963]: Starting upload Aug 16 15:41:12.402160428 (190h14m53.133428058s) [upload=197963]: Done starting upload Aug 16 15:41:12.402246779 (190h14m53.1335144s) [upload=197963]: Uploading segments... Aug 16 15:41:12.402480159 (190h14m53.133747789s) [upload=197963]: Got next segment. Inline: false Aug 16 15:41:12.402508461 (190h14m53.133776082s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:41:12.402512469 (190h14m53.133780089s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:41:12.432979783 (190h14m53.164247433s) [seg_pos={0 0}, upload=197963]: Begin upload segment... Aug 16 15:41:12.433069851 (190h14m53.164337502s) [seg_pos={0 0}, upload=197963]: Done begin upload segment. Aug 16 15:41:12.433300475 (190h14m53.164568125s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.433371609 (190h14m53.164639269s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.434684907 (190h14m53.165952566s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.435481013 (190h14m53.166748662s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.436348603 (190h14m53.167616253s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.437054379 (190h14m53.16832203s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.440286702 (190h14m53.171554363s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.445732877 (190h14m53.177000497s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.447246201 (190h14m53.178513851s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.448284202 (190h14m53.179551832s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.449463608 (190h14m53.180731238s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450094433 (190h14m53.181362063s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450145950 (190h14m53.18141357s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.450200563 (190h14m53.181468183s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451444931 (190h14m53.182712551s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.451774661 (190h14m53.18304228s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Uploading piece... Aug 16 15:41:12.534048827 (190h14m53.265316477s) [upload=197963]: Next returned nil segment Aug 16 15:41:12.534088210 (190h14m53.265355861s) [upload=197963]: Waiting for error group managing segments... Aug 16 15:41:12.548971298 (190h14m53.280238948s) [noise=true , address=104.36.82.15:4257 , piece_id=H4FXNRR2DVAZB2UZBJTMMDSDQK5KIVH7BUZQRFR3ARQS5SKHYATA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.550856190 (190h14m53.28212381s) [noise=true , address=104.36.82.7:4910 , piece_id=SJNRRSBANKU45SJVFTIX2CV5S2QQE7YYZPKWLS7DPE5D4SWZCVPA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.590344154 (190h14m53.321611804s) [noise=true , address=104.36.82.6:4876 , piece_id=ZXYRDJ47L5C2E7JCYNSMXNN3OQGWGTJZOK4H36OTGXBILOWQGMYA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.601114663 (190h14m53.332382293s) [noise=true , address=104.36.82.12:4151 , piece_id=VBGUSFCI6M6ZPHVEJ5NUE64VJGZTVKODJXXKCUIWK6ZSYAPKXXGA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:12.823455396 (190h14m53.554723036s) [noise=true , address=77.90.31.15:4698 , piece_id=T2YHKVVZNSWEPXDBEQHLGIJTH2FBEIUXIOJ2F3XOZ64BRRCZ6HJA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.416489395 (190h14m54.147757015s) [noise=true , address=45.91.116.6:4974 , piece_id=KDFOS7U7JUILPHQGL5TTARYUYCU67I7A2VF52YMQG6RHKKJWP3QA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:13.532032808 (190h14m54.263300438s) [noise=true , address=45.91.116.11:4119 , piece_id=EP427WIGG7C52GF5KCPTZLI5H3GVGC5FOM5YJLZNOGSDADLWFK5Q, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.035449985 (190h14m54.766717605s) [noise=true , address=77.90.31.13:4626 , piece_id=O4JLXXU6VJVMBDO2C6PTTGBAP2XPN7F6H44WVKE7YA2M2VGKKHAQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.103439759 (190h14m54.834707389s) [noise=true , address=45.91.116.4:4940 , piece_id=IPI3YEWLAZZYSJJWE2Y3Y6TWUGXSAVUFJGUVHBMZAB7PL4RQHCXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.347174953 (190h14m55.078442583s) [noise=true , address=45.91.116.13:4155 , piece_id=K7DWZJUFF4A54KTYZK6GDQV5MQTRWL6CQHHC7AO6S6HM5UQGYDBA, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:14.439594001 (190h14m55.170861631s) [noise=true , address=45.91.116.12:4227 , piece_id=TIEX2JV7TULV73J6ECGWWF4IRJ5T7NMRT72DTGPFXCS2LOWFMZXQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:36.322085073 (190h15m17.053352703s) [noise=true , address=104.36.82.3:4767 , piece_id=3QTVTENB4N57OK4LXIRGEQ3GQ5L2MB5PVSIWCWRO5EQJ2JIREGKQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072321103 (190h15m21.803588722s) [noise=true , address=104.36.82.16:4293 , piece_id=TND7VYCXYY3WKHOTZA3KEBSBCJQAGVUW3I5GXN54DWNG2NH3WNZQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: <nil> Aug 16 15:41:41.072335469 (190h15m21.803603099s) [seg_pos={0 0}, upload=197963]: Segment reached optimal threshold of 13 pieces. Aug 16 15:41:41.072678443 (190h15m21.803946073s) [noise=true , address=77.90.31.21:4878 , piece_id=YYKY7ME3GAYPYFRY4G4LWDYOB63WPPRB5QZXFHGIIUDCZL73GKMQ, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: rpc: dial tcp 77.90.31.21:4878: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072720493 (190h15m21.803988123s) [noise=true , address=104.36.82.9:4007 , piece_id=CDZBRNZBS3NHADILYDNG3XL6MNUGA6BP3EV3YJHLRSEITS2PIB6A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:39758->104.36.82.9:4007: use of closed network connection Aug 16 15:41:41.072750890 (190h15m21.804018511s) [noise=true , address=77.90.31.21:4877 , piece_id=ZET5MRUHU4RPHN6HPF7HR3LHJD62T5HPA56UQZTTUD2NWKM7TJ7A, seg_pos={0 0}, upload=197963]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: read tcp 5.161.89.252:12140->77.90.31.21:4877: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:41:41.072758936 (190h15m21.804026566s) [seg_pos={0 0}, upload=197963]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 16 15:41:41.072806925 (190h15m21.804074555s) [upload=197963]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 16 15:41:41.072810192 (190h15m21.804077812s) [upload=197963]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:41:41.249494107 (190h15m21.980761756s) [upload=197963]: Done uploading segments... Aug 16 15:41:41.249514585 (190h15m21.980782235s) [upload=197963]: Upload finished. err: <nil> Duration: 30.001786646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X.................XX.........X.........X.........] duration: 21.670625322s FAIL (2025-08-16 15:41:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 21.670656278s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X...X............................] duration: 757.548768ms FAIL (2025-08-16 15:41:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 757.569185ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X...X...........................] duration: 1.372187746s FAIL (2025-08-16 15:40:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after add180cd317e136e9a7269306da283467a21eadd86dcadb7122496ee78413e50 != before b700f26ab6ceef98f22cf7e1ab637c0c58f804be339381b2834069199fdb5975 ======LOGS====== Aug 16 15:21:20.817605457 (189h55m1.548873088s) [upload=197614]: Starting upload Aug 16 15:21:20.817644491 (189h55m1.548912111s) [upload=197614]: Done starting upload Aug 16 15:21:20.817690889 (189h55m1.548958508s) [upload=197614]: Uploading segments... Aug 16 15:21:20.817726545 (189h55m1.548994165s) [upload=197614]: Got next segment. Inline: true Aug 16 15:21:20.817730163 (189h55m1.548997793s) [upload=197614]: Waiting for error group managing segments... Aug 16 15:21:20.817763114 (189h55m1.549030744s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:21:20.817766591 (189h55m1.54903421s) [upload=197614]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 16 15:21:20.817769385 (189h55m1.549037016s) [upload=197614]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 16 15:21:32.010025302 (189h55m12.741292962s) [upload=197614]: Done uploading segments... Aug 16 15:21:32.010051571 (189h55m12.741319231s) [upload=197614]: Upload finished. err: <nil> Duration:* 1.372210568s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [............X.................XX.........X.........X........] duration: 10.511017856s FAIL (2025-08-16 15:40:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 16 15:34:26.287594313 (190h8m7.018861964s) [upload=197841]: Starting upload Aug 16 15:34:26.287673282 (190h8m7.018940932s) [upload=197841]: Done starting upload Aug 16 15:34:26.287760015 (190h8m7.019027665s) [upload=197841]: Uploading segments... Aug 16 15:34:26.288099383 (190h8m7.019367033s) [upload=197841]: Got next segment. Inline: false Aug 16 15:34:26.288216152 (190h8m7.019483801s) [upload=197841]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:34:26.288222844 (190h8m7.019490494s) [upload=197841]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:34:26.886756649 (190h8m7.618024279s) [seg_pos={0 0}, upload=197841]: Begin upload segment... Aug 16 15:34:26.886788890 (190h8m7.61805652s) [seg_pos={0 0}, upload=197841]: Done begin upload segment. Aug 16 15:34:26.886816923 (190h8m7.618084542s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.886825118 (190h8m7.618092738s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887075188 (190h8m7.618342798s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887096758 (190h8m7.618364388s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887298477 (190h8m7.618566097s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887342901 (190h8m7.618610531s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887591587 (190h8m7.618859237s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887725810 (190h8m7.61899346s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.887983744 (190h8m7.619251395s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888213156 (190h8m7.619480806s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888377043 (190h8m7.619644694s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888639517 (190h8m7.619907166s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.888760415 (190h8m7.620028064s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889137533 (190h8m7.620405183s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889149996 (190h8m7.620417646s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889582560 (190h8m7.620850219s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889640809 (190h8m7.620908459s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:26.889990375 (190h8m7.621258025s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Uploading piece... Aug 16 15:34:27.011584069 (190h8m7.742851718s) [upload=197841]: Next returned nil segment Aug 16 15:34:27.011598355 (190h8m7.742866006s) [upload=197841]: Waiting for error group managing segments... Aug 16 15:34:27.079793204 (190h8m7.811060854s) [noise=true , address=77.90.31.16:4745 , piece_id=WV33JYRQ7UBIOCXY2VSE5RJEGV2SVSOMLWWPWV5BMGRIKWR2C37A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.121889549 (190h8m7.853157199s) [noise=true , address=77.90.31.15:4708 , piece_id=BEYFRJ3BYZBFT2V6PUUJOBCEDCQ4QXRSLSRYVV3S57APO3KI6ISA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.135396561 (190h8m7.86666421s) [noise=true , address=77.90.31.14:4665 , piece_id=PIYIU3UACZ2KWZ5PMQVEPJLFV6X4Q6YERJPATMHPGZEE52CDFNVA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.421703313 (190h8m8.152970964s) [noise=true , address=45.91.116.11:4131 , piece_id=E7XL33TLGATD6MV5DPPQWBQBSLNLEOMC4WQJN4TTLDC63GKVVU2A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:27.738756936 (190h8m8.470024566s) [noise=true , address=45.91.116.5:4307 , piece_id=P7U7YMU4QEOWOVDSX63T3QPWSZ3SFY27FQYB7OLHN55BMYFXAC4A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ATcyutSAE1A1uN2QcwYFdY1ZvUCZTKiybzXt66NgnqrFYoWWC, address:45.91.116.5:4307): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:28.985799854 (190h8m9.717067484s) [noise=true , address=66.23.193.175:4559 , piece_id=5DQRSCJQQJWKI5F6ZLRCDWRPE2CRVHLFEFXGGLTWGX6KFOX36PYA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:30.174181181 (190h8m10.905448851s) [noise=true , address=45.91.116.12:4237 , piece_id=EK5ZT6F3E5ZPOSIEY2IWVVPJA4WITBBZGWHLCUF7SK4MSOINKBIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:32.145934082 (190h8m12.877201732s) [noise=true , address=45.91.116.6:4999 , piece_id=SEKW2OFVU53PPBDN5JELKTONJXXCYRYW5YRXPDH76SWTWYQI2RAA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:12evc8vF5f1Z7cPrrC7G7BsGuDWjzttAE2FsQciV7nztBDmrCZB, address:45.91.116.6:4999): protocol: storage node overloaded, request limit: 2000 Aug 16 15:34:34.486935619 (190h8m15.218203269s) [noise=true , address=79.127.220.101:4061 , piece_id=XPBWCFRKWFVQZCZM7AOUUZZDZRRHD2POGMYOAERGCWR5MEREDUKQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.408572801 (190h8m17.139840451s) [noise=true , address=104.36.82.44:4881 , piece_id=T65GDSARQ3RCE4N43E64GOVGWBQOK4XKGGQMUAHSQTWNPIGOGVSQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:36.892474253 (190h8m17.623741903s) [noise=true , address=77.90.31.18:4822 , piece_id=4XFI6KEYMW7B5OOK6AZ3DELBPQSQ6L6YCNAS6MUG6PU6BSP2PGIQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:41.598275361 (190h8m22.32954301s) [noise=true , address=104.36.82.58:4266 , piece_id=Q6BIUXOPA72Q2FUO2HJWMBTRYFDF5EJUOAI2XIXHRWFGVBPPLY6Q, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:43.463190476 (190h8m24.194458126s) [noise=true , address=104.36.82.2:4752 , piece_id=3XZP235LZRTP2OHICT7LJTZNDVECHHBO4W2QEWKK23VFP6MK7JJQ, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:34:44.381727093 (190h8m25.112994743s) [noise=true , address=104.36.82.60:4351 , piece_id=CUSUZ7DSMOJXMCUNR4GUDS6AU47PKKQIRSOCCFNYXOWJNVNDZ6RA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:02.431209734 (190h8m43.162477384s) [noise=true , address=104.36.82.23:4479 , piece_id=2P3BCTIHTKEKJ3DIEIBAJ5FREH6Q4PS6DZFGUN5BOADNZFNFMOXA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: <nil> Aug 16 15:35:26.269944636 (190h9m7.001212286s) [noise=true , address=104.36.82.5:4853 , piece_id=D46JRZGEJMRLK3BY2266LHZUHQB7XIZAFS3SUSMQWAEWBCXVQ6TA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1ycsU8APac3LBkfY1xCsc8b9KJQVAx5kFJnYebWhKSKSbKyf5u, address:104.36.82.5:4853): protocol: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:19100->104.36.82.5:4853: use of closed network connection Aug 16 15:35:26.270108384 (190h9m7.001376034s) [noise=true , address=77.90.31.11:4568 , piece_id=PYUENPYYDKRPTHF4GHVIRT6ABQLLMW2MBNP7PIXPU7OMBWPDV23A, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1H1L8DTKD7W7Xx1PVT7KzB9GkGBadnTEj9BHtbSgcW1S7Jj8Tw, address:77.90.31.11:4568): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:35:26.270127429 (190h9m7.00139509s) [noise=true , address=77.90.31.19:4848 , piece_id=C4EQUIQABL63YSTLI5MTTR35MU33VQQUSOHIY2EOJ4B425KO6THA, seg_pos={0 0}, upload=197841]: Done uploading piece. err: ecclient: upload failed (node:1EFoU2X25nensqiGn1NwTk1LcVDFK9yeJaGBm1nDTe8GqfGXwD, address:77.90.31.19:4848): protocol: write tcp 5.161.89.252:45938->77.90.31.19:4848: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 16 15:35:26.270152938 (190h9m7.001420587s) [seg_pos={0 0}, upload=197841]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 16 15:35:27.945901312 (190h9m8.677169012s) [upload=197841]: Done uploading segments... Aug 16 15:35:27.945924786 (190h9m8.677192416s) [upload=197841]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 10.511057701s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XX.X......] duration: 8.468589358s FAIL (2025-08-16 15:40:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Aug 16 15:36:55.548879704 (190h10m36.280147324s) [upload=197887]: Starting upload Aug 16 15:36:55.548955528 (190h10m36.280223147s) [upload=197887]: Done starting upload Aug 16 15:36:55.549016942 (190h10m36.280284563s) [upload=197887]: Uploading segments... Aug 16 15:36:55.549162556 (190h10m36.280430175s) [upload=197887]: Got next segment. Inline: false Aug 16 15:36:55.549208583 (190h10m36.280476213s) [upload=197887]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 16 15:36:55.549212680 (190h10m36.2804803s) [upload=197887]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 16 15:36:55.572770508 (190h10m36.304038128s) [seg_pos={0 0}, upload=197887]: Begin upload segment... Aug 16 15:36:55.572812216 (190h10m36.304079846s) [seg_pos={0 0}, upload=197887]: Done begin upload segment. Aug 16 15:36:55.572837433 (190h10m36.304105064s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.572888521 (190h10m36.304156151s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573150964 (190h10m36.304418594s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573396756 (190h10m36.304664376s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573405983 (190h10m36.304673603s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573662676 (190h10m36.304930295s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573827505 (190h10m36.305095124s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.573888319 (190h10m36.305155949s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574169407 (190h10m36.305437037s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574283202 (190h10m36.305550821s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574401553 (190h10m36.305669183s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574663826 (190h10m36.305931456s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.574673755 (190h10m36.305941375s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575045013 (190h10m36.306312633s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.575976403 (190h10m36.307244023s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.576318816 (190h10m36.307586446s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Uploading piece... Aug 16 15:36:55.655755600 (190h10m36.38702322s) [upload=197887]: Next returned nil segment Aug 16 15:36:55.655775477 (190h10m36.387043098s) [upload=197887]: Waiting for error group managing segments... Aug 16 15:36:55.666571205 (190h10m36.397838825s) [noise=true , address=104.36.82.11:4114 , piece_id=4TC4UYJ6NQS7FIAYFC7NRHGPN7Q5IEVJD56CYU4W73VJX5MUMGEA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668360567 (190h10m36.399628187s) [noise=true , address=104.36.82.6:4874 , piece_id=DBQZFQELTGVZFDVGGTZ3T3P5W37SSHD3IXL7EPQKA5QDB7XM4YUQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.668816834 (190h10m36.400084454s) [noise=true , address=104.36.82.7:4909 , piece_id=WGXTE2INRIC3S6OKBMKFLDUBWZNLVIWQQHEETZYFRFDUFJ62YYPQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.676071787 (190h10m36.407339407s) [noise=true , address=104.36.82.15:4259 , piece_id=B42P6SMMBWEOVPZD3H6YKFGWSTEI26MM477YGVRAIU4FH4SUANEQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.678258677 (190h10m36.409526307s) [noise=true , address=45.91.116.12:4227 , piece_id=NBNZYCC3WRCPEFH5RBYOT2GDWGNVJSXQVKCVICWZFDPDOUXMZSDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.683863309 (190h10m36.41513095s) [noise=true , address=45.91.116.5:4297 , piece_id=ABC2QVUVUQSBQSFQ7EC6KPOXVJTG7KCTVS4PTVYZV5PS4FCFI5IA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.685196184 (190h10m36.416463814s) [noise=true , address=77.90.31.14:4663 , piece_id=7PIRTI6Z2YIA7672NMBG7NXGMWJ6RWLXAIDZKGRBEVVFR3A7ZJMQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.692404148 (190h10m36.423671769s) [noise=true , address=104.36.82.12:4150 , piece_id=OGMMOCHENTDWX7PKYPLMVRKGMPPLQ3YJYHYBIF4DHCWSJQ7NWNQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.698148323 (190h10m36.429415943s) [noise=true , address=77.90.31.21:4877 , piece_id=H2MDECKUVJHZ7LHC7UEAFWP4KNV5TWUWHOFYQJZ5LDX7AFL5IWZQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.717275295 (190h10m36.448542936s) [noise=true , address=77.90.31.12:4592 , piece_id=YIYYEGHEB6DX7Y5ZDGE4NGRYLU7RHBTOES5YLAQIXAD3RQSFTJ3A, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.752694974 (190h10m36.483962594s) [noise=true , address=45.91.116.10:4082 , piece_id=I3NHXLFT3LAMHQOZGXRKF3ZMQTQG6VMJA4BFCZCVLBRVLLBZWWQQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:36:55.821678736 (190h10m36.552946355s) [noise=true , address=104.36.82.11:4113 , piece_id=WLE3K7KBSXZJ7ATZ4VGB4L33IRCTTANMTLECJM3MAL7GA4F6E4OA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: <nil> Aug 16 15:37:25.532779122 (190h11m6.264046772s) [noise=true , address=77.90.31.18:4807 , piece_id=OUCGRCJ6I4A3DADK4FO24UZSFRQBZDA7TL43CC4TXVU4YIJ74LVA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.532793719 (190h11m6.264061369s) [noise=true , address=104.36.82.2:4729 , piece_id=DPYQQSPLPLQ3F4XCR3XCPY7M2WMFZCKEO2SGCMND6HJLFNLXUPOQ, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:104.36.82.2:4729): protocol: expected piece hash; context deadline exceeded; EOF Aug 16 15:37:25.532949402 (190h11m6.264217052s) [noise=true , address=104.36.82.4:4801 , piece_id=TFFHWD66MWXIVQIKQ2RIQCYXGHO655TM3GO5HILAVZR6LTGL7ZDA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:104.36.82.4:4801): protocol: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:21554->104.36.82.4:4801: use of closed network connection Aug 16 15:37:25.533151812 (190h11m6.264419463s) [noise=true , address=77.90.31.15:4699 , piece_id=E4IL6FMEPI2RRCFFKQ67ENGNYSPOU3FZWW7LB3SKE5B2CMK22PIA, seg_pos={0 0}, upload=197887]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: context deadline exceeded; context deadline exceeded Aug 16 15:37:25.533188301 (190h11m6.264455951s) [seg_pos={0 0}, upload=197887]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Aug 16 15:37:25.547075237 (190h11m6.278342897s) [upload=197887]: Done uploading segments... Aug 16 15:37:25.547090556 (190h11m6.278358206s) [upload=197887]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.468615668s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)