Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
316.549429ms
FAIL (2025-11-30 10:03:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 316.569716ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.....................................................XXXXX..] duration:
566.711948ms
FAIL (2025-11-30 10:03:25 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 566.732538ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [......................................................XXXXX.] duration:
16.401872431s
FAIL (2025-11-30 10:03:09 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 16.401884464s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
343.894846ms
FAIL (2025-11-30 10:03:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 343.911547ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
5 out of 60 is failed (threshold=3) [......................................................XXXXX.] duration:
23.636338161s
FAIL (2025-11-30 10:03:05 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
165596 (814h48m8.109110511s) [seg_pos={0 0}, upload=716889]: Done begin upload segment.
Nov 30 10:01:41.408194700 (814h48m8.109139616s) [noise=true , address=145.79.227.5:4235 , piece_id=HUAIZFMCVATSUHT2B7M62H6CUZ6MVZFCBKAJ6U6QWX62LG632RAQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.408224707 (814h48m8.109169622s) [noise=true , address=45.91.116.12:4247 , piece_id=SJCO2XP7AUZUN3JLM5NHX5DRDWMR6TBVXTMWSSX53KYHRDIBDYIA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.408698056 (814h48m8.109642971s) [noise=true , address=77.90.31.13:4637 , piece_id=BXAN7NRCXCMNORCJRUDS6PR3MSOZ4MJD3HZOLD4PS7WNB63WHBKA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.408704358 (814h48m8.109649272s) [noise=true , address=45.91.116.10:4104 , piece_id=RIGUL7KSO2JK5M7TNDVKQZD6FLZDEWYDXZRFHEWILUNBZTJYUW2Q, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.409120197 (814h48m8.110065113s) [noise=true , address=77.90.31.18:4812 , piece_id=MRSFSFJXTW5GXC4XHFR6Q2X3QSGXKGT2O4TL32JFCDMXZFJ6SVDA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.409130086 (814h48m8.110075001s) [noise=true , address=64.49.13.154:4934 , piece_id=HJVI7UG2JQCOP2TV4BDKJUDFT5X55TFFQVC3O47AETFPEJHZEXBQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.409504428 (814h48m8.110449343s) [noise=true , address=145.79.227.13:4807 , piece_id=VU4WQ423XOJWRKJRMKEKVJSMP3QLR5HYQG3GMJS7ZEJDHYILULAQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.409543391 (814h48m8.110488316s) [noise=true , address=64.49.13.170:4717 , piece_id=QFYCR6LHBUW277EU62RII3WYKCWH6NCMMBJ72MHQZJI3UUFOS5GQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.410342341 (814h48m8.111287255s) [noise=true , address=77.90.31.10:4548 , piece_id=NFF3IVBCV6UPVW3OXTP7WIO6IIQJZTSFXJXZCZVNR2HBCRH4EH3A, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.410374791 (814h48m8.111319756s) [noise=true , address=77.90.31.21:4891 , piece_id=JPRZXCFQO4VCYJJY5PV6MET4ZF64NBU7CWMAGISCBTYFXVLARPBQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.410970980 (814h48m8.111915896s) [noise=true , address=64.49.13.169:4716 , piece_id=DBG2O7KRPUVWYKWDNQI5VKBUL46FTJUBJP5ZWDHWCVZAPKCXQAPA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.412047490 (814h48m8.112992406s) [noise=true , address=45.91.116.7:4034 , piece_id=3TGU5SC64E3SJURS2FJNBMNHH6HCTIZUGP6XZGZNJDSH5YHHZVNA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.412199065 (814h48m8.11314398s) [noise=true , address=45.91.116.14:4192 , piece_id=USKGGDF5655GRJ2JX6JZFGWRR6JME3KMQGRI6PFJH42HLIV4RVYQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.413134330 (814h48m8.114079245s) [noise=true , address=77.90.31.17:4794 , piece_id=2A533NKQB2W4X3EXNBHE3M3THMDOT77QKYLHNMJXCBYNYFILNHYA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.413280624 (814h48m8.114225539s) [noise=true , address=77.90.31.11:4574 , piece_id=KOAG4T3FABZHOUKKKHU3D5HAUAXZPIZ7OPKNX3XHHHDAI3KILYQQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.414080115 (814h48m8.11502503s) [noise=true , address=77.90.31.14:4668 , piece_id=WJEEZHF4QOWYMHQCW3V43E6OO63ELRZJ5E3XLI3KIIXV2LLF4WGA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.414219066 (814h48m8.115163981s) [noise=true , address=45.91.116.9:4286 , piece_id=LJ5HJED3MRX7PBBPVUIIFPQCXKOZYUUYNKTTU6QFC3TJ2VKRCRDQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.414913329 (814h48m8.115858243s) [noise=true , address=45.91.116.13:4162 , piece_id=QFNCJIWNSQUT2VYQGBKZZTHU7YM3XO33XZYZI5O7FT3WQBK36T3Q, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.497117191 (814h48m8.198062086s) [upload=716889]: Next returned nil segment
Nov 30 10:01:41.497127249 (814h48m8.198072144s) [upload=716889]: Waiting for error group managing segments...
Nov 30 10:01:41.505081407 (814h48m8.206026291s) [noise=true , address=45.91.116.10:4104 , piece_id=RIGUL7KSO2JK5M7TNDVKQZD6FLZDEWYDXZRFHEWILUNBZTJYUW2Q, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.505294346 (814h48m8.206239231s) [noise=true , address=45.91.116.12:4247 , piece_id=SJCO2XP7AUZUN3JLM5NHX5DRDWMR6TBVXTMWSSX53KYHRDIBDYIA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.505597595 (814h48m8.206542521s) [noise=true , address=45.91.116.7:4034 , piece_id=3TGU5SC64E3SJURS2FJNBMNHH6HCTIZUGP6XZGZNJDSH5YHHZVNA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.505940729 (814h48m8.206885644s) [noise=true , address=45.91.116.9:4286 , piece_id=LJ5HJED3MRX7PBBPVUIIFPQCXKOZYUUYNKTTU6QFC3TJ2VKRCRDQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.506595328 (814h48m8.207540253s) [noise=true , address=145.79.227.13:4807 , piece_id=VU4WQ423XOJWRKJRMKEKVJSMP3QLR5HYQG3GMJS7ZEJDHYILULAQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.507083102 (814h48m8.208028018s) [noise=true , address=145.79.227.5:4235 , piece_id=HUAIZFMCVATSUHT2B7M62H6CUZ6MVZFCBKAJ6U6QWX62LG632RAQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.507415977 (814h48m8.208360892s) [noise=true , address=45.91.116.13:4162 , piece_id=QFNCJIWNSQUT2VYQGBKZZTHU7YM3XO33XZYZI5O7FT3WQBK36T3Q, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.520800485 (814h48m8.22174537s) [noise=true , address=45.91.116.14:4192 , piece_id=USKGGDF5655GRJ2JX6JZFGWRR6JME3KMQGRI6PFJH42HLIV4RVYQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.974912342 (814h48m8.675857227s) [noise=true , address=64.49.13.170:4717 , piece_id=QFYCR6LHBUW277EU62RII3WYKCWH6NCMMBJ72MHQZJI3UUFOS5GQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.997957398 (814h48m8.698902293s) [noise=true , address=64.49.13.154:4934 , piece_id=HJVI7UG2JQCOP2TV4BDKJUDFT5X55TFFQVC3O47AETFPEJHZEXBQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:42.860910619 (814h48m9.561855514s) [noise=true , address=64.49.13.169:4716 , piece_id=DBG2O7KRPUVWYKWDNQI5VKBUL46FTJUBJP5ZWDHWCVZAPKCXQAPA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:02:41.372086706 (814h49m8.073031601s) [noise=true , address=77.90.31.10:4548 , piece_id=NFF3IVBCV6UPVW3OXTP7WIO6IIQJZTSFXJXZCZVNR2HBCRH4EH3A, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: rpc: dial tcp 77.90.31.10:4548: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372182455 (814h49m8.073127341s) [noise=true , address=77.90.31.13:4637 , piece_id=BXAN7NRCXCMNORCJRUDS6PR3MSOZ4MJD3HZOLD4PS7WNB63WHBKA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17, address:77.90.31.13:4637): protocol: write tcp 5.161.89.252:54802->77.90.31.13:4637: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372220236 (814h49m8.073165131s) [noise=true , address=77.90.31.11:4574 , piece_id=KOAG4T3FABZHOUKKKHU3D5HAUAXZPIZ7OPKNX3XHHHDAI3KILYQQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: rpc: dial tcp 77.90.31.11:4574: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372279337 (814h49m8.073224233s) [noise=true , address=77.90.31.17:4794 , piece_id=2A533NKQB2W4X3EXNBHE3M3THMDOT77QKYLHNMJXCBYNYFILNHYA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:1uouoLKpbu62sxnPGb5c5U8Y8z4tAYBTzLFFHdA51VLrFSZc63, address:77.90.31.17:4794): protocol: write tcp 5.161.89.252:23588->77.90.31.17:4794: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372309224 (814h49m8.073254119s) [noise=true , address=77.90.31.21:4891 , piece_id=JPRZXCFQO4VCYJJY5PV6MET4ZF64NBU7CWMAGISCBTYFXVLARPBQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: rpc: dial tcp 77.90.31.21:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372345802 (814h49m8.073290697s) [noise=true , address=77.90.31.18:4812 , piece_id=MRSFSFJXTW5GXC4XHFR6Q2X3QSGXKGT2O4TL32JFCDMXZFJ6SVDA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:128qNJJSSwMBPvN8wmTcUeMWAJxVmpjWjwA6WTKxBNCmT2FdFJn, address:77.90.31.18:4812): protocol: rpc: dial tcp 77.90.31.18:4812: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372411244 (814h49m8.07335613s) [noise=true , address=77.90.31.14:4668 , piece_id=WJEEZHF4QOWYMHQCW3V43E6OO63ELRZJ5E3XLI3KIIXV2LLF4WGA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:1U4SKog2kLXNyjyWs7Jg1MGerbhSYLwJUbskE3TGCHJpNWQHS7, address:77.90.31.14:4668): protocol: write tcp 5.161.89.252:11036->77.90.31.14:4668: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372429860 (814h49m8.073374745s) [seg_pos={0 0}, upload=716889]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Nov 30 10:02:41.388787807 (814h49m8.089732692s) [upload=716889]: Done uploading segments...
Nov 30 10:02:41.388801623 (814h49m8.089746508s) [upload=716889]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 23.636352838s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=3) [...................................................XXXXXXXX.] duration:
1.462128842s
FAIL (2025-11-30 10:03:04 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.462142208s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
1m0.001348231s
FAIL (2025-11-30 10:02:52 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001361685s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration:
5.001507517s
FAIL (2025-11-30 10:02:46 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001528957s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
313.217798ms
FAIL (2025-11-30 10:02:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 313.230312ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
1m0.018802049s
FAIL (2025-11-30 10:02:41 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
165596 (814h48m8.109110511s) [seg_pos={0 0}, upload=716889]: Done begin upload segment.
Nov 30 10:01:41.408194700 (814h48m8.109139616s) [noise=true , address=145.79.227.5:4235 , piece_id=HUAIZFMCVATSUHT2B7M62H6CUZ6MVZFCBKAJ6U6QWX62LG632RAQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.408224707 (814h48m8.109169622s) [noise=true , address=45.91.116.12:4247 , piece_id=SJCO2XP7AUZUN3JLM5NHX5DRDWMR6TBVXTMWSSX53KYHRDIBDYIA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.408698056 (814h48m8.109642971s) [noise=true , address=77.90.31.13:4637 , piece_id=BXAN7NRCXCMNORCJRUDS6PR3MSOZ4MJD3HZOLD4PS7WNB63WHBKA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.408704358 (814h48m8.109649272s) [noise=true , address=45.91.116.10:4104 , piece_id=RIGUL7KSO2JK5M7TNDVKQZD6FLZDEWYDXZRFHEWILUNBZTJYUW2Q, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.409120197 (814h48m8.110065113s) [noise=true , address=77.90.31.18:4812 , piece_id=MRSFSFJXTW5GXC4XHFR6Q2X3QSGXKGT2O4TL32JFCDMXZFJ6SVDA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.409130086 (814h48m8.110075001s) [noise=true , address=64.49.13.154:4934 , piece_id=HJVI7UG2JQCOP2TV4BDKJUDFT5X55TFFQVC3O47AETFPEJHZEXBQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.409504428 (814h48m8.110449343s) [noise=true , address=145.79.227.13:4807 , piece_id=VU4WQ423XOJWRKJRMKEKVJSMP3QLR5HYQG3GMJS7ZEJDHYILULAQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.409543391 (814h48m8.110488316s) [noise=true , address=64.49.13.170:4717 , piece_id=QFYCR6LHBUW277EU62RII3WYKCWH6NCMMBJ72MHQZJI3UUFOS5GQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.410342341 (814h48m8.111287255s) [noise=true , address=77.90.31.10:4548 , piece_id=NFF3IVBCV6UPVW3OXTP7WIO6IIQJZTSFXJXZCZVNR2HBCRH4EH3A, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.410374791 (814h48m8.111319756s) [noise=true , address=77.90.31.21:4891 , piece_id=JPRZXCFQO4VCYJJY5PV6MET4ZF64NBU7CWMAGISCBTYFXVLARPBQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.410970980 (814h48m8.111915896s) [noise=true , address=64.49.13.169:4716 , piece_id=DBG2O7KRPUVWYKWDNQI5VKBUL46FTJUBJP5ZWDHWCVZAPKCXQAPA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.412047490 (814h48m8.112992406s) [noise=true , address=45.91.116.7:4034 , piece_id=3TGU5SC64E3SJURS2FJNBMNHH6HCTIZUGP6XZGZNJDSH5YHHZVNA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.412199065 (814h48m8.11314398s) [noise=true , address=45.91.116.14:4192 , piece_id=USKGGDF5655GRJ2JX6JZFGWRR6JME3KMQGRI6PFJH42HLIV4RVYQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.413134330 (814h48m8.114079245s) [noise=true , address=77.90.31.17:4794 , piece_id=2A533NKQB2W4X3EXNBHE3M3THMDOT77QKYLHNMJXCBYNYFILNHYA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.413280624 (814h48m8.114225539s) [noise=true , address=77.90.31.11:4574 , piece_id=KOAG4T3FABZHOUKKKHU3D5HAUAXZPIZ7OPKNX3XHHHDAI3KILYQQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.414080115 (814h48m8.11502503s) [noise=true , address=77.90.31.14:4668 , piece_id=WJEEZHF4QOWYMHQCW3V43E6OO63ELRZJ5E3XLI3KIIXV2LLF4WGA, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.414219066 (814h48m8.115163981s) [noise=true , address=45.91.116.9:4286 , piece_id=LJ5HJED3MRX7PBBPVUIIFPQCXKOZYUUYNKTTU6QFC3TJ2VKRCRDQ, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.414913329 (814h48m8.115858243s) [noise=true , address=45.91.116.13:4162 , piece_id=QFNCJIWNSQUT2VYQGBKZZTHU7YM3XO33XZYZI5O7FT3WQBK36T3Q, seg_pos={0 0}, upload=716889]: Uploading piece...
Nov 30 10:01:41.497117191 (814h48m8.198062086s) [upload=716889]: Next returned nil segment
Nov 30 10:01:41.497127249 (814h48m8.198072144s) [upload=716889]: Waiting for error group managing segments...
Nov 30 10:01:41.505081407 (814h48m8.206026291s) [noise=true , address=45.91.116.10:4104 , piece_id=RIGUL7KSO2JK5M7TNDVKQZD6FLZDEWYDXZRFHEWILUNBZTJYUW2Q, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.505294346 (814h48m8.206239231s) [noise=true , address=45.91.116.12:4247 , piece_id=SJCO2XP7AUZUN3JLM5NHX5DRDWMR6TBVXTMWSSX53KYHRDIBDYIA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.505597595 (814h48m8.206542521s) [noise=true , address=45.91.116.7:4034 , piece_id=3TGU5SC64E3SJURS2FJNBMNHH6HCTIZUGP6XZGZNJDSH5YHHZVNA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.505940729 (814h48m8.206885644s) [noise=true , address=45.91.116.9:4286 , piece_id=LJ5HJED3MRX7PBBPVUIIFPQCXKOZYUUYNKTTU6QFC3TJ2VKRCRDQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.506595328 (814h48m8.207540253s) [noise=true , address=145.79.227.13:4807 , piece_id=VU4WQ423XOJWRKJRMKEKVJSMP3QLR5HYQG3GMJS7ZEJDHYILULAQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.507083102 (814h48m8.208028018s) [noise=true , address=145.79.227.5:4235 , piece_id=HUAIZFMCVATSUHT2B7M62H6CUZ6MVZFCBKAJ6U6QWX62LG632RAQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.507415977 (814h48m8.208360892s) [noise=true , address=45.91.116.13:4162 , piece_id=QFNCJIWNSQUT2VYQGBKZZTHU7YM3XO33XZYZI5O7FT3WQBK36T3Q, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.520800485 (814h48m8.22174537s) [noise=true , address=45.91.116.14:4192 , piece_id=USKGGDF5655GRJ2JX6JZFGWRR6JME3KMQGRI6PFJH42HLIV4RVYQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.974912342 (814h48m8.675857227s) [noise=true , address=64.49.13.170:4717 , piece_id=QFYCR6LHBUW277EU62RII3WYKCWH6NCMMBJ72MHQZJI3UUFOS5GQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:41.997957398 (814h48m8.698902293s) [noise=true , address=64.49.13.154:4934 , piece_id=HJVI7UG2JQCOP2TV4BDKJUDFT5X55TFFQVC3O47AETFPEJHZEXBQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:01:42.860910619 (814h48m9.561855514s) [noise=true , address=64.49.13.169:4716 , piece_id=DBG2O7KRPUVWYKWDNQI5VKBUL46FTJUBJP5ZWDHWCVZAPKCXQAPA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: <nil>
Nov 30 10:02:41.372086706 (814h49m8.073031601s) [noise=true , address=77.90.31.10:4548 , piece_id=NFF3IVBCV6UPVW3OXTP7WIO6IIQJZTSFXJXZCZVNR2HBCRH4EH3A, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: rpc: dial tcp 77.90.31.10:4548: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372182455 (814h49m8.073127341s) [noise=true , address=77.90.31.13:4637 , piece_id=BXAN7NRCXCMNORCJRUDS6PR3MSOZ4MJD3HZOLD4PS7WNB63WHBKA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17, address:77.90.31.13:4637): protocol: write tcp 5.161.89.252:54802->77.90.31.13:4637: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372220236 (814h49m8.073165131s) [noise=true , address=77.90.31.11:4574 , piece_id=KOAG4T3FABZHOUKKKHU3D5HAUAXZPIZ7OPKNX3XHHHDAI3KILYQQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: rpc: dial tcp 77.90.31.11:4574: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372279337 (814h49m8.073224233s) [noise=true , address=77.90.31.17:4794 , piece_id=2A533NKQB2W4X3EXNBHE3M3THMDOT77QKYLHNMJXCBYNYFILNHYA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:1uouoLKpbu62sxnPGb5c5U8Y8z4tAYBTzLFFHdA51VLrFSZc63, address:77.90.31.17:4794): protocol: write tcp 5.161.89.252:23588->77.90.31.17:4794: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372309224 (814h49m8.073254119s) [noise=true , address=77.90.31.21:4891 , piece_id=JPRZXCFQO4VCYJJY5PV6MET4ZF64NBU7CWMAGISCBTYFXVLARPBQ, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:17RDAerfx8AW5NsgWLTXLNDbuAEWQFT3SVM4iwdmuBzWrML83g, address:77.90.31.21:4891): protocol: rpc: dial tcp 77.90.31.21:4891: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372345802 (814h49m8.073290697s) [noise=true , address=77.90.31.18:4812 , piece_id=MRSFSFJXTW5GXC4XHFR6Q2X3QSGXKGT2O4TL32JFCDMXZFJ6SVDA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:128qNJJSSwMBPvN8wmTcUeMWAJxVmpjWjwA6WTKxBNCmT2FdFJn, address:77.90.31.18:4812): protocol: rpc: dial tcp 77.90.31.18:4812: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372411244 (814h49m8.07335613s) [noise=true , address=77.90.31.14:4668 , piece_id=WJEEZHF4QOWYMHQCW3V43E6OO63ELRZJ5E3XLI3KIIXV2LLF4WGA, seg_pos={0 0}, upload=716889]: Done uploading piece. err: ecclient: upload failed (node:1U4SKog2kLXNyjyWs7Jg1MGerbhSYLwJUbskE3TGCHJpNWQHS7, address:77.90.31.14:4668): protocol: write tcp 5.161.89.252:11036->77.90.31.14:4668: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:02:41.372429860 (814h49m8.073374745s) [seg_pos={0 0}, upload=716889]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Nov 30 10:02:41.388787807 (814h49m8.089732692s) [upload=716889]: Done uploading segments...
Nov 30 10:02:41.388801623 (814h49m8.089746508s) [upload=716889]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 1m0.018814712s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
373.657729ms
FAIL (2025-11-30 10:02:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 373.682516ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
404.611716ms
FAIL (2025-11-30 10:02:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 404.631144ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
354.265338ms
FAIL (2025-11-30 10:02:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 354.283843ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
5.003767429s
FAIL (2025-11-30 10:01:58 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.003779081s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
1m0.000853419s
FAIL (2025-11-30 10:01:52 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000867214s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
312.755021ms
FAIL (2025-11-30 10:01:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 312.772933ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
1m0.019225824s
FAIL (2025-11-30 10:01:41 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
) [noise=true , address=77.90.31.10:4533 , piece_id=TXGVDKUYKK4XAFYDNO557JCTBYG26XL4NQQPZQTKMH2FEBC3U4JA, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.338090780 (814h47m8.039035675s) [noise=true , address=45.91.116.5:4313 , piece_id=SN7EMT4H4LGBF6YZZFXM7YQB6QRX633K2AND6AXO4ULNIL53EYQA, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.338427241 (814h47m8.039372127s) [noise=true , address=45.91.116.14:4195 , piece_id=2YAOBHN3UM3OBJFZYK6EPE3LMJ6THLLZYTE3JZ7N24WC24E27K3A, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.338698891 (814h47m8.039643777s) [noise=true , address=145.79.227.6:4265 , piece_id=HMYYB5H4GNGZNZPO3GGFNPF67IOTKDJQHNGB3ABMAMI6CQKTURKQ, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.338991470 (814h47m8.039936365s) [noise=true , address=77.90.31.17:4777 , piece_id=KZYSQR6K4FTGLZS3WNC6X6UTQLOFXB3XNYHXLHOZYIIUPEBEWNJA, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.339261197 (814h47m8.040206092s) [noise=true , address=77.90.31.11:4579 , piece_id=3CPMNAZIJ6FZKBEHSOLX7LM6VY23NX6N6745664HMT3A5NI6AEKA, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.339270404 (814h47m8.040215299s) [noise=true , address=77.90.31.14:4665 , piece_id=55GZ4MPYSKGXPPIODSLQGCUPXCBPGHSGSVZGROP6WTNFP2WHZ52A, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.339526314 (814h47m8.04047121s) [noise=true , address=64.49.13.169:4707 , piece_id=RK7Y5EVQJ3PX3GUT6YROBWXZPTJVLHXNXWVPGGHAY34225XTD5QA, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.339536974 (814h47m8.040481859s) [noise=true , address=77.90.31.12:4600 , piece_id=G53TDNJDSBWGDXSEOECD2UWDGYUU4D6RA5G43BLEVC6C7CYKL43Q, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.339788817 (814h47m8.040733712s) [noise=true , address=45.91.116.13:4178 , piece_id=NBXMHMNH5HWRKIQ7RHUXD556MXBON4HUCZ44AULXVTZGM6KCL2UA, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.339804856 (814h47m8.040749752s) [noise=true , address=77.90.31.21:4909 , piece_id=KIUYRVQW4GTKNFDQQD3FAV5HZELAPMH5PYG3GG2BJ4QOQZYNV27A, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.340103839 (814h47m8.041048734s) [noise=true , address=145.79.227.4:4212 , piece_id=CLSB3RD2CUA4IRMPNXTNFAR7Z4MGY6WUJCMHH5PMU7BV6OS4IQLA, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.340114037 (814h47m8.041058923s) [noise=true , address=45.91.116.10:4087 , piece_id=3RGLXCY7M2A4MM54CLVY7KD26ZAAB5FODLULHTSZF56UJHABVFMQ, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.340384825 (814h47m8.04132972s) [noise=true , address=77.90.31.16:4739 , piece_id=OMNBC5C73MQSXKGMIPL2N2SXPOIGFPWFWRNKM6WOTQ7TDWOMXRNQ, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.340655974 (814h47m8.041600869s) [noise=true , address=45.91.116.9:4265 , piece_id=ALNUNA6YDWUTBJFDWHJZIXZKZ2O2ITH2MPEJZQMZKCH5XN5B77VQ, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.340956729 (814h47m8.041901624s) [noise=true , address=145.79.227.15:4886 , piece_id=6GZJYGDBEP7MRCR3EA2YXSYFJXGSDRXNB6Y2PVCAQFBM3ASBFCLA, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.341235051 (814h47m8.042179936s) [noise=true , address=64.49.13.154:4936 , piece_id=HT4UT735EMSQ6KN7AVGMVNOTRECUCZR5D67VFYUDBRMT57TNBBHQ, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.341666279 (814h47m8.042611165s) [noise=true , address=77.90.31.19:4871 , piece_id=3HS4CCZVISGHR43P3TNZJLPGUKZERONFOC3PRUCHEQ5OILXT6PAQ, seg_pos={0 0}, upload=716876]: Uploading piece...
Nov 30 10:00:41.416004742 (814h47m8.116949647s) [upload=716876]: Next returned nil segment
Nov 30 10:00:41.416045218 (814h47m8.116990123s) [upload=716876]: Waiting for error group managing segments...
Nov 30 10:00:41.425048093 (814h47m8.125992988s) [noise=true , address=45.91.116.13:4178 , piece_id=NBXMHMNH5HWRKIQ7RHUXD556MXBON4HUCZ44AULXVTZGM6KCL2UA, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:41.425254450 (814h47m8.126199335s) [noise=true , address=45.91.116.9:4265 , piece_id=ALNUNA6YDWUTBJFDWHJZIXZKZ2O2ITH2MPEJZQMZKCH5XN5B77VQ, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:41.425597213 (814h47m8.126542108s) [noise=true , address=145.79.227.15:4886 , piece_id=6GZJYGDBEP7MRCR3EA2YXSYFJXGSDRXNB6Y2PVCAQFBM3ASBFCLA, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:41.425930439 (814h47m8.126875314s) [noise=true , address=145.79.227.6:4265 , piece_id=HMYYB5H4GNGZNZPO3GGFNPF67IOTKDJQHNGB3ABMAMI6CQKTURKQ, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:41.434850387 (814h47m8.135795282s) [noise=true , address=45.91.116.10:4087 , piece_id=3RGLXCY7M2A4MM54CLVY7KD26ZAAB5FODLULHTSZF56UJHABVFMQ, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:41.435658343 (814h47m8.136603238s) [noise=true , address=45.91.116.5:4313 , piece_id=SN7EMT4H4LGBF6YZZFXM7YQB6QRX633K2AND6AXO4ULNIL53EYQA, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:41.465784017 (814h47m8.166728902s) [noise=true , address=145.79.227.4:4212 , piece_id=CLSB3RD2CUA4IRMPNXTNFAR7Z4MGY6WUJCMHH5PMU7BV6OS4IQLA, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:41.863430433 (814h47m8.564375328s) [noise=true , address=45.91.116.14:4195 , piece_id=2YAOBHN3UM3OBJFZYK6EPE3LMJ6THLLZYTE3JZ7N24WC24E27K3A, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:41.922902606 (814h47m8.623847501s) [noise=true , address=64.49.13.154:4936 , piece_id=HT4UT735EMSQ6KN7AVGMVNOTRECUCZR5D67VFYUDBRMT57TNBBHQ, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:00:43.081139896 (814h47m9.782084791s) [noise=true , address=64.49.13.169:4707 , piece_id=RK7Y5EVQJ3PX3GUT6YROBWXZPTJVLHXNXWVPGGHAY34225XTD5QA, seg_pos={0 0}, upload=716876]: Done uploading piece. err: <nil>
Nov 30 10:01:41.300658310 (814h48m8.001603206s) [noise=true , address=77.90.31.12:4600 , piece_id=G53TDNJDSBWGDXSEOECD2UWDGYUU4D6RA5G43BLEVC6C7CYKL43Q, seg_pos={0 0}, upload=716876]: Done uploading piece. err: ecclient: upload failed (node:12PKT2EcU6jAe6oUUdiTcwxGQsFa36uEmown77SemqQMM5Xw5nc, address:77.90.31.12:4600): protocol: rpc: dial tcp 77.90.31.12:4600: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:01:41.300701130 (814h48m8.001646016s) [noise=true , address=77.90.31.16:4739 , piece_id=OMNBC5C73MQSXKGMIPL2N2SXPOIGFPWFWRNKM6WOTQ7TDWOMXRNQ, seg_pos={0 0}, upload=716876]: Done uploading piece. err: ecclient: upload failed (node:129B3N7yjJvwCzoVLTqWsPbcLQ5TQUyAzg1RCkbSAVKPWYTG8eb, address:77.90.31.16:4739): protocol: rpc: dial tcp 77.90.31.16:4739: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:01:41.300768727 (814h48m8.001713622s) [noise=true , address=77.90.31.19:4871 , piece_id=3HS4CCZVISGHR43P3TNZJLPGUKZERONFOC3PRUCHEQ5OILXT6PAQ, seg_pos={0 0}, upload=716876]: Done uploading piece. err: ecclient: upload failed (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm, address:77.90.31.19:4871): protocol: rpc: dial tcp 77.90.31.19:4871: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:01:41.300786050 (814h48m8.001730945s) [noise=true , address=77.90.31.21:4909 , piece_id=KIUYRVQW4GTKNFDQQD3FAV5HZELAPMH5PYG3GG2BJ4QOQZYNV27A, seg_pos={0 0}, upload=716876]: Done uploading piece. err: ecclient: upload failed (node:121RT7tP6czMFAiZ3DjzSSsYgPt3xZXVbXVpkcYxaZYscNiMjL6, address:77.90.31.21:4909): protocol: write tcp 5.161.89.252:41098->77.90.31.21:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:01:41.300894073 (814h48m8.001838957s) [noise=true , address=77.90.31.14:4665 , piece_id=55GZ4MPYSKGXPPIODSLQGCUPXCBPGHSGSVZGROP6WTNFP2WHZ52A, seg_pos={0 0}, upload=716876]: Done uploading piece. err: ecclient: upload failed (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce, address:77.90.31.14:4665): protocol: context deadline exceeded; context deadline exceeded
Nov 30 10:01:41.300944016 (814h48m8.001888901s) [noise=true , address=77.90.31.11:4579 , piece_id=3CPMNAZIJ6FZKBEHSOLX7LM6VY23NX6N6745664HMT3A5NI6AEKA, seg_pos={0 0}, upload=716876]: Done uploading piece. err: ecclient: upload failed (node:12dRcuBLuT2ybWGzdkp4vyo7e37qrZk6SBpWWHty5WKRoVHp1p, address:77.90.31.11:4579): protocol: write tcp 5.161.89.252:64610->77.90.31.11:4579: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:01:41.300960517 (814h48m8.001905412s) [noise=true , address=77.90.31.10:4533 , piece_id=TXGVDKUYKK4XAFYDNO557JCTBYG26XL4NQQPZQTKMH2FEBC3U4JA, seg_pos={0 0}, upload=716876]: Done uploading piece. err: ecclient: upload failed (node:1RDgg3m5Febnz5CQKgkHvVYKBAzJG341rrT6pMchR1XxxEb7KS, address:77.90.31.10:4533): protocol: rpc: dial tcp 77.90.31.10:4533: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:01:41.301020289 (814h48m8.001965184s) [noise=true , address=77.90.31.17:4777 , piece_id=KZYSQR6K4FTGLZS3WNC6X6UTQLOFXB3XNYHXLHOZYIIUPEBEWNJA, seg_pos={0 0}, upload=716876]: Done uploading piece. err: ecclient: upload failed (node:1DjhLCRQRkNECPQg8HG8kZcBn8jJmBZpNhdywxEK7ZUn1v19wM, address:77.90.31.17:4777): protocol: write tcp 5.161.89.252:50166->77.90.31.17:4777: use of closed network connection; context deadline exceeded
Nov 30 10:01:41.301038773 (814h48m8.001983669s) [seg_pos={0 0}, upload=716876]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
Nov 30 10:01:41.318108617 (814h48m8.019053512s) [upload=716876]: Done uploading segments...
Nov 30 10:01:41.318125779 (814h48m8.019070664s) [upload=716876]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 1m0.019236834s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
314.200573ms
FAIL (2025-11-30 10:01:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 314.216242ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
5.000915096s
FAIL (2025-11-30 10:01:08 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000929162s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
339.515238ms
FAIL (2025-11-30 10:01:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 339.534294ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
282.850241ms
FAIL (2025-11-30 10:00:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 282.867633ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
1m0.000606233s
FAIL (2025-11-30 10:00:52 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000617844s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
1m0.015755883s
FAIL (2025-11-30 10:00:41 UTC)
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 12)
======LOGS======
Got next segment. Inline: false
Nov 30 09:59:41.245256744 (814h46m7.946201669s) [upload=716860]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:59:41.245262746 (814h46m7.946207671s) [upload=716860]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:59:41.265285029 (814h46m7.966229943s) [seg_pos={0 0}, upload=716860]: Begin upload segment...
Nov 30 09:59:41.265352757 (814h46m7.966297682s) [seg_pos={0 0}, upload=716860]: Done begin upload segment.
Nov 30 09:59:41.265391489 (814h46m7.966336464s) [noise=true , address=45.91.116.10:4115 , piece_id=CXFGBLORTLNEFLC23FIW3AKAW5GQ2I4KAQWT3UQK3HMAUG6YJORQ, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.265410725 (814h46m7.96635566s) [noise=true , address=145.79.227.2:4144 , piece_id=KWWHDAU6XEX7CBZCVHVQSFNKG5TPVMULL5J735PYT5BR7CKXRAYA, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.265488070 (814h46m7.966432995s) [noise=true , address=45.91.116.13:4184 , piece_id=EDCJCHJRSWURXUGF2GQDM67G2E5T5SFUKLA5XH257B3DZDHFRO3Q, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.266143079 (814h46m7.967088004s) [noise=true , address=45.91.116.6:4979 , piece_id=2UCTLBA5FJ247RJ2A6RWKFTRMC4TNLBGJ676L3NVRWRVMWJUVFNQ, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.266759276 (814h46m7.967704201s) [noise=true , address=64.49.13.138:4482 , piece_id=7ZZ47EJZXJXJFMU3F7WB4SCUUXAI2IHCWBCCZ2Z4VC4A3FXGE5BA, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.267362278 (814h46m7.968307202s) [noise=true , address=77.90.31.18:4813 , piece_id=FU6FFO5SNFADAOG3MHCEDCMGMQLO2E4C5RVOLKY4KTFQSD5IHAFA, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.268009672 (814h46m7.968954597s) [noise=true , address=77.90.31.10:4548 , piece_id=BUZ7J3KP4IQBBFXT2XL5PEW2RY4C32QKTCCC7TLRM6V5IZFOEIWQ, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.268651557 (814h46m7.969596482s) [noise=true , address=45.91.116.8:4059 , piece_id=K6I5VV35XXBZ6CRPGVI2IEQMUGFURCKEJG3AKVU7ETKJM77HWTFQ, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.269292730 (814h46m7.970237655s) [noise=true , address=77.90.31.21:4880 , piece_id=D32FNLHBSL4JKUH6UHWJZCBFIZBXV3PZD6HXR4HUP56UDFSYDONA, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.270546373 (814h46m7.971491297s) [noise=true , address=45.91.116.12:4230 , piece_id=ILFRHCASFPNS7FGKQJOAGKAPIQIUO6IVP2UNNTPNEH4QT4LYH5MA, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.271091856 (814h46m7.972036782s) [noise=true , address=64.49.13.169:4710 , piece_id=NIHVIQBKLH4RTBMNUZEH4PIYSQRNTUUY3EX5Z7GKZHW6ZJ2XQGJA, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.271560705 (814h46m7.972505631s) [noise=true , address=145.79.227.3:4168 , piece_id=7RR2MK473BZEG4M353KTNVXVQKFKDRXGXLNF7GO77NZ445PN744A, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.272147767 (814h46m7.973092693s) [noise=true , address=77.90.31.15:4708 , piece_id=QP4FAJFP246ALXPSYFGIRTHEYGXHXMONEBRN6MPNUXRFLGNPWLPQ, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.272870785 (814h46m7.97381571s) [noise=true , address=77.90.31.17:4777 , piece_id=PUOERILVET6HJMKE5TXAG4ADOJLZHF2IZ5DMPS2CFSBEMSLOIQGQ, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.273683629 (814h46m7.974628544s) [noise=true , address=45.91.116.11:4143 , piece_id=DQN5JKRT27JVDJ4COIUWRWJPUU5FDEMKTV4N3OMLVJIPBTX6PYNA, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.274470817 (814h46m7.975415742s) [noise=true , address=145.79.227.11:4737 , piece_id=SDGMBOSJZZYJQRL6I4VUVPPCWEQ765TE5UPT3YCP6HPSESFIUSHQ, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.275266339 (814h46m7.976211255s) [noise=true , address=64.49.13.155:4916 , piece_id=Z5KBBSAFV4UTIJBSWFEFD2RWT37NBCSKISGPJF5DTPHX5X5IGG6A, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.276887502 (814h46m7.977832427s) [noise=true , address=77.90.31.12:4614 , piece_id=G4PA3VZZFKZ4EMZOTARSWS4RNY7OAJS3KRBJW35YCBF6CKOJHH4Q, seg_pos={0 0}, upload=716860]: Uploading piece...
Nov 30 09:59:41.380212460 (814h46m8.081157375s) [upload=716860]: Next returned nil segment
Nov 30 09:59:41.380226196 (814h46m8.081171111s) [upload=716860]: Waiting for error group managing segments...
Nov 30 09:59:41.388175653 (814h46m8.089120568s) [noise=true , address=45.91.116.6:4979 , piece_id=2UCTLBA5FJ247RJ2A6RWKFTRMC4TNLBGJ676L3NVRWRVMWJUVFNQ, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.388935158 (814h46m8.089880074s) [noise=true , address=45.91.116.11:4143 , piece_id=DQN5JKRT27JVDJ4COIUWRWJPUU5FDEMKTV4N3OMLVJIPBTX6PYNA, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.389029585 (814h46m8.089974501s) [noise=true , address=45.91.116.12:4230 , piece_id=ILFRHCASFPNS7FGKQJOAGKAPIQIUO6IVP2UNNTPNEH4QT4LYH5MA, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.389459092 (814h46m8.090404018s) [noise=true , address=45.91.116.13:4184 , piece_id=EDCJCHJRSWURXUGF2GQDM67G2E5T5SFUKLA5XH257B3DZDHFRO3Q, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.389917672 (814h46m8.090862587s) [noise=true , address=145.79.227.11:4737 , piece_id=SDGMBOSJZZYJQRL6I4VUVPPCWEQ765TE5UPT3YCP6HPSESFIUSHQ, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.390048788 (814h46m8.090993713s) [noise=true , address=45.91.116.8:4059 , piece_id=K6I5VV35XXBZ6CRPGVI2IEQMUGFURCKEJG3AKVU7ETKJM77HWTFQ, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.390417310 (814h46m8.091362225s) [noise=true , address=145.79.227.2:4144 , piece_id=KWWHDAU6XEX7CBZCVHVQSFNKG5TPVMULL5J735PYT5BR7CKXRAYA, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.406661484 (814h46m8.107606408s) [noise=true , address=145.79.227.3:4168 , piece_id=7RR2MK473BZEG4M353KTNVXVQKFKDRXGXLNF7GO77NZ445PN744A, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.409626357 (814h46m8.110571272s) [noise=true , address=45.91.116.10:4115 , piece_id=CXFGBLORTLNEFLC23FIW3AKAW5GQ2I4KAQWT3UQK3HMAUG6YJORQ, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.865304804 (814h46m8.566249728s) [noise=true , address=64.49.13.155:4916 , piece_id=Z5KBBSAFV4UTIJBSWFEFD2RWT37NBCSKISGPJF5DTPHX5X5IGG6A, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:41.869916488 (814h46m8.570861403s) [noise=true , address=64.49.13.138:4482 , piece_id=7ZZ47EJZXJXJFMU3F7WB4SCUUXAI2IHCWBCCZ2Z4VC4A3FXGE5BA, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 09:59:42.069687916 (814h46m8.770632831s) [noise=true , address=64.49.13.169:4710 , piece_id=NIHVIQBKLH4RTBMNUZEH4PIYSQRNTUUY3EX5Z7GKZHW6ZJ2XQGJA, seg_pos={0 0}, upload=716860]: Done uploading piece. err: <nil>
Nov 30 10:00:41.230665358 (814h47m7.931610253s) [noise=true , address=77.90.31.10:4548 , piece_id=BUZ7J3KP4IQBBFXT2XL5PEW2RY4C32QKTCCC7TLRM6V5IZFOEIWQ, seg_pos={0 0}, upload=716860]: Done uploading piece. err: ecclient: upload failed (node:1pj8fDt6E3oe2jUTDBrfN3wL7KVuJymqmGf5Y1x4tozmGfTcr7, address:77.90.31.10:4548): protocol: rpc: dial tcp 77.90.31.10:4548: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:00:41.230765605 (814h47m7.9317105s) [noise=true , address=77.90.31.21:4880 , piece_id=D32FNLHBSL4JKUH6UHWJZCBFIZBXV3PZD6HXR4HUP56UDFSYDONA, seg_pos={0 0}, upload=716860]: Done uploading piece. err: ecclient: upload failed (node:1vQ1WWxf2CL7wRKRsKsRsWc4nkCbD1h8tRLsA3QYYAft91yUb, address:77.90.31.21:4880): protocol: rpc: dial tcp 77.90.31.21:4880: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:00:41.230789791 (814h47m7.931734686s) [noise=true , address=77.90.31.15:4708 , piece_id=QP4FAJFP246ALXPSYFGIRTHEYGXHXMONEBRN6MPNUXRFLGNPWLPQ, seg_pos={0 0}, upload=716860]: Done uploading piece. err: ecclient: upload failed (node:12cGT35JePb3CDVwWxdHPJDPBdeLuLvMSm8Xdmx2KRMavTTbY6u, address:77.90.31.15:4708): protocol: rpc: dial tcp 77.90.31.15:4708: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:00:41.230816331 (814h47m7.931761216s) [noise=true , address=77.90.31.17:4777 , piece_id=PUOERILVET6HJMKE5TXAG4ADOJLZHF2IZ5DMPS2CFSBEMSLOIQGQ, seg_pos={0 0}, upload=716860]: Done uploading piece. err: ecclient: upload failed (node:1DjhLCRQRkNECPQg8HG8kZcBn8jJmBZpNhdywxEK7ZUn1v19wM, address:77.90.31.17:4777): protocol: rpc: dial tcp 77.90.31.17:4777: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 10:00:41.230852759 (814h47m7.931797644s) [noise=true , address=77.90.31.18:4813 , piece_id=FU6FFO5SNFADAOG3MHCEDCMGMQLO2E4C5RVOLKY4KTFQSD5IHAFA, seg_pos={0 0}, upload=716860]: Done uploading piece. err: ecclient: upload failed (node:128WthA1x9wp1dV1vkpGnoveZQpU9CSgDCZfw5GMQuVpCNSVAM5, address:77.90.31.18:4813): protocol: context deadline exceeded; context deadline exceeded
Nov 30 10:00:41.230919935 (814h47m7.93186482s) [noise=true , address=77.90.31.12:4614 , piece_id=G4PA3VZZFKZ4EMZOTARSWS4RNY7OAJS3KRBJW35YCBF6CKOJHH4Q, seg_pos={0 0}, upload=716860]: Done uploading piece. err: ecclient: upload failed (node:12wKLBwTAdUVAQmQGcafdpUcjiDE89jxdptEwXoAvV6WXf2Xbc3, address:77.90.31.12:4614): protocol: write tcp 5.161.89.252:58198->77.90.31.12:4614: use of closed network connection; context deadline exceeded
Nov 30 10:00:41.230935885 (814h47m7.93188078s) [seg_pos={0 0}, upload=716860]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil>
Nov 30 10:00:41.245021037 (814h47m7.945965923s) [upload=716860]: Done uploading segments...
Nov 30 10:00:41.245039803 (814h47m7.945984698s) [upload=716860]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12)
Duration: 1m0.015772684s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
363.897923ms
FAIL (2025-11-30 10:00:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 363.924013ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
5.002458342s
FAIL (2025-11-30 10:00:26 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.002477557s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
5.00053358s
FAIL (2025-11-30 10:00:08 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000547537s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
437.701293ms
FAIL (2025-11-30 10:00:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 437.718445ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
319.444393ms
FAIL (2025-11-30 10:00:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 319.612229ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
1m0.000762661s
FAIL (2025-11-30 09:59:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000774433s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
1m0.019098442s
FAIL (2025-11-30 09:59:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
_id=3JA7UFGRLW47JST5XLHXG32JRGOZDONE3V5YP645VBXDKHO53ZVA, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.202821860 (814h45m7.903766775s) [noise=true , address=77.90.31.21:4909 , piece_id=ZT4C36ZXAYP32LQIHZK4RBII6KG5WYCX37V4NHVZJL3Q5ATMZI5A, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.203364739 (814h45m7.904309664s) [noise=true , address=145.79.227.3:4182 , piece_id=MNL5XMOWLINZ7TCW2GP475JJWJ6ZXH7QLFR5GHGQIMHDFVY4P3FQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.203890776 (814h45m7.904835691s) [noise=true , address=45.91.116.8:4053 , piece_id=ZQD6I6ESWSCT37MJ6JWHGQFZQMKZGIFZ6Q3YVAQW6IVRPHHS6AKA, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.204351150 (814h45m7.905296066s) [noise=true , address=64.49.13.169:4714 , piece_id=7J7D6MBZMPCMEFPNEP3YXYQ5UNKVFBMFURSBQZ4L6KGXRGYMFLUQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.204754407 (814h45m7.905699332s) [noise=true , address=145.79.227.5:4242 , piece_id=M2QJDVMFB7I3ORB4M2DC4UX4OPQBAVBS3LWVPNRPTHYBJXXUDFDQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.205176840 (814h45m7.906121755s) [noise=true , address=45.91.116.6:4994 , piece_id=L4SBCPVN3MQ2X2CU2TDUZ2ZKOFCERY7SCOIKV2ECS76ORYQ2JCIA, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.205584414 (814h45m7.90652933s) [noise=true , address=45.91.116.4:4957 , piece_id=BL7O4SV22HLLYVX3U2OP46MLXTXM6ZJXONR6R4VLEUJQ3NR5DYGQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.206035020 (814h45m7.906979936s) [noise=true , address=64.49.13.137:4498 , piece_id=PMCX266KB7OWNO4IGQG3EVHOTJIMV26G2ZYYXB5ZXPX7JAMPZQFQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.206585703 (814h45m7.907530618s) [noise=true , address=77.90.31.12:4607 , piece_id=DHPVVTNETZ6ZB4V2KETKYHNH2LJZNESSRRQELIO4JZIAYMZIGVVQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.208167912 (814h45m7.909112798s) [noise=true , address=77.90.31.11:4571 , piece_id=SP4HBFP326ZIM6JDAJHIFTIAZTF4G4XK3A3HHPOUPL7CIOY3KLPA, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.209294697 (814h45m7.910239592s) [noise=true , address=77.90.31.10:4536 , piece_id=3FRI3PDXLJSZLU6CPTBWX5FKXEZBD7I6K5QEKEDSEG6GTTI54GAQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.211670296 (814h45m7.91261521s) [noise=true , address=77.90.31.19:4857 , piece_id=TIGXRF7LHYU7VFHT4FNQK4RDAANAOG3B5CQZG5K7JFYVNR4OWZNQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.213884320 (814h45m7.914829235s) [noise=true , address=145.79.227.8:4362 , piece_id=SPE6VMRI4OYDONWDTAPJCK4M2CIRA2B5UDHHA6MIMTCTHFIIETJA, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.216498326 (814h45m7.91744324s) [noise=true , address=77.90.31.14:4692 , piece_id=V4FUFC4QVPMYG6A57P5OCQNBQNU5XKV635XTXC3ELEJ42Q75UNSA, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.217663021 (814h45m7.918607906s) [noise=true , address=77.90.31.15:4714 , piece_id=Y5VNOZ4HKWEZGD5BKEB7ZUTNXS2VPVDWWAUM2QL2OMDIY4T7HOQA, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.218907005 (814h45m7.9198519s) [noise=true , address=45.91.116.7:4025 , piece_id=X6I4EKCB733YHTNLEIKVPCU7S3GCKYTFQ22DFS2LL3NNGIBDD3BA, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.220098071 (814h45m7.921042956s) [noise=true , address=77.90.31.13:4635 , piece_id=SNQBLWDSEBI34NCTQJ7GDLHJRYWSRMUOIZY3OOQSZZJ2T3P2UKFQ, seg_pos={0 0}, upload=716846]: Uploading piece...
Nov 30 09:58:41.279345694 (814h45m7.980290578s) [upload=716846]: Next returned nil segment
Nov 30 09:58:41.279370099 (814h45m7.980314994s) [upload=716846]: Waiting for error group managing segments...
Nov 30 09:58:41.289446839 (814h45m7.990391734s) [noise=true , address=45.91.116.7:4025 , piece_id=X6I4EKCB733YHTNLEIKVPCU7S3GCKYTFQ22DFS2LL3NNGIBDD3BA, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:41.289811753 (814h45m7.990756679s) [noise=true , address=145.79.227.14:4848 , piece_id=3JA7UFGRLW47JST5XLHXG32JRGOZDONE3V5YP645VBXDKHO53ZVA, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:41.289964850 (814h45m7.990909735s) [noise=true , address=45.91.116.8:4053 , piece_id=ZQD6I6ESWSCT37MJ6JWHGQFZQMKZGIFZ6Q3YVAQW6IVRPHHS6AKA, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:41.291155986 (814h45m7.992100911s) [noise=true , address=145.79.227.8:4362 , piece_id=SPE6VMRI4OYDONWDTAPJCK4M2CIRA2B5UDHHA6MIMTCTHFIIETJA, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:41.292032721 (814h45m7.992977606s) [noise=true , address=145.79.227.5:4242 , piece_id=M2QJDVMFB7I3ORB4M2DC4UX4OPQBAVBS3LWVPNRPTHYBJXXUDFDQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:41.293405838 (814h45m7.994350753s) [noise=true , address=45.91.116.6:4994 , piece_id=L4SBCPVN3MQ2X2CU2TDUZ2ZKOFCERY7SCOIKV2ECS76ORYQ2JCIA, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:41.305764560 (814h45m8.006709445s) [noise=true , address=45.91.116.4:4957 , piece_id=BL7O4SV22HLLYVX3U2OP46MLXTXM6ZJXONR6R4VLEUJQ3NR5DYGQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:41.308863806 (814h45m8.009808721s) [noise=true , address=145.79.227.3:4182 , piece_id=MNL5XMOWLINZ7TCW2GP475JJWJ6ZXH7QLFR5GHGQIMHDFVY4P3FQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:41.779146204 (814h45m8.480091129s) [noise=true , address=64.49.13.137:4498 , piece_id=PMCX266KB7OWNO4IGQG3EVHOTJIMV26G2ZYYXB5ZXPX7JAMPZQFQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:58:43.069155957 (814h45m9.770100872s) [noise=true , address=64.49.13.169:4714 , piece_id=7J7D6MBZMPCMEFPNEP3YXYQ5UNKVFBMFURSBQZ4L6KGXRGYMFLUQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: <nil>
Nov 30 09:59:41.163546038 (814h46m7.864490964s) [noise=true , address=77.90.31.14:4692 , piece_id=V4FUFC4QVPMYG6A57P5OCQNBQNU5XKV635XTXC3ELEJ42Q75UNSA, seg_pos={0 0}, upload=716846]: Done uploading piece. err: ecclient: upload failed (node:1ckv57Jo1CoAnLarXfwwQB2qmUNxjVuNksEpiXchpPVGxLfw3T, address:77.90.31.14:4692): protocol: rpc: dial tcp 77.90.31.14:4692: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:59:41.163683937 (814h46m7.864628852s) [noise=true , address=77.90.31.21:4909 , piece_id=ZT4C36ZXAYP32LQIHZK4RBII6KG5WYCX37V4NHVZJL3Q5ATMZI5A, seg_pos={0 0}, upload=716846]: Done uploading piece. err: ecclient: upload failed (node:121RT7tP6czMFAiZ3DjzSSsYgPt3xZXVbXVpkcYxaZYscNiMjL6, address:77.90.31.21:4909): protocol: rpc: dial tcp 77.90.31.21:4909: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:59:41.163791559 (814h46m7.864736484s) [noise=true , address=77.90.31.15:4714 , piece_id=Y5VNOZ4HKWEZGD5BKEB7ZUTNXS2VPVDWWAUM2QL2OMDIY4T7HOQA, seg_pos={0 0}, upload=716846]: Done uploading piece. err: ecclient: upload failed (node:12DTMJVUhaPKvEnvujCAk9byGb88QsevLbtCBB9R1mxS4DxShiP, address:77.90.31.15:4714): protocol: rpc: dial tcp 77.90.31.15:4714: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:59:41.163980373 (814h46m7.864925288s) [noise=true , address=77.90.31.11:4571 , piece_id=SP4HBFP326ZIM6JDAJHIFTIAZTF4G4XK3A3HHPOUPL7CIOY3KLPA, seg_pos={0 0}, upload=716846]: Done uploading piece. err: ecclient: upload failed (node:12eshfvC8guQXYqSeUqwmycyBbM4ens9W98cktmW7nPzk1D6pwy, address:77.90.31.11:4571): protocol: rpc: dial tcp 77.90.31.11:4571: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:59:41.164025408 (814h46m7.864970322s) [noise=true , address=77.90.31.13:4635 , piece_id=SNQBLWDSEBI34NCTQJ7GDLHJRYWSRMUOIZY3OOQSZZJ2T3P2UKFQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: ecclient: upload failed (node:1hkgeCi3efNQFv37chVFEi4v2AyWFEwv9keyy5oz6RkJ4Ubunp, address:77.90.31.13:4635): protocol: write tcp 5.161.89.252:25968->77.90.31.13:4635: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:59:41.164144841 (814h46m7.865089757s) [noise=true , address=77.90.31.10:4536 , piece_id=3FRI3PDXLJSZLU6CPTBWX5FKXEZBD7I6K5QEKEDSEG6GTTI54GAQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: ecclient: upload failed (node:1zwuzCLWy2q166Ld52WUG9fvXNdJWsQrg2NearHYg7pyxpXFen, address:77.90.31.10:4536): protocol: write tcp 5.161.89.252:46166->77.90.31.10:4536: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:59:41.164205906 (814h46m7.865150831s) [noise=true , address=77.90.31.19:4857 , piece_id=TIGXRF7LHYU7VFHT4FNQK4RDAANAOG3B5CQZG5K7JFYVNR4OWZNQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: ecclient: upload failed (node:12i57FF97oEddYLZpNxSq5or9WLEquL8ZW1cVUuW4AThBbDqcCL, address:77.90.31.19:4857): protocol: rpc: dial tcp 77.90.31.19:4857: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:59:41.164249679 (814h46m7.865194594s) [noise=true , address=77.90.31.12:4607 , piece_id=DHPVVTNETZ6ZB4V2KETKYHNH2LJZNESSRRQELIO4JZIAYMZIGVVQ, seg_pos={0 0}, upload=716846]: Done uploading piece. err: ecclient: upload failed (node:12UYNy47GZEha8wG9vZNUAws5ogTBjSXLdt7T2bwqcHGsPrKFrm, address:77.90.31.12:4607): protocol: rpc: dial tcp 77.90.31.12:4607: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:59:41.164276470 (814h46m7.865221384s) [seg_pos={0 0}, upload=716846]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
Nov 30 09:59:41.179417552 (814h46m7.880362508s) [upload=716846]: Done uploading segments...
Nov 30 09:59:41.179435045 (814h46m7.880379981s) [upload=716846]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 1m0.019111666s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
5.000929765s
FAIL (2025-11-30 09:59:29 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000965241s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
371.293328ms
FAIL (2025-11-30 09:59:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 371.318817ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
336.797605ms
FAIL (2025-11-30 09:58:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 336.946654ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.000799567s
FAIL (2025-11-30 09:58:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.00083284s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.01698899s
FAIL (2025-11-30 09:58:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
_id=2APJUPF5QSXVE2BCBDSXZVXG6LGOOBN6EGNGJTACE7UHDVRJE2TA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.124778848 (814h44m7.825723743s) [noise=true , address=77.90.31.14:4665 , piece_id=V6E4QAFOP33HX2GH5UGMJNYAIHFORO5BXNL7JTITMLQ43WEA7R3A, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.124880830 (814h44m7.825825725s) [noise=true , address=64.49.13.137:4495 , piece_id=JXC26YQ2P3GI6JMDSZBFCFCH7KU2NHLTXXQ7TJ34QK4GSW45XKKQ, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.125140387 (814h44m7.826085271s) [noise=true , address=77.90.31.21:4899 , piece_id=YRENXKSRXUOQS7O7A54DJVPYO4XFKKHH76LVPHLIURRMICUC7CYA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.125376590 (814h44m7.826321485s) [noise=true , address=77.90.31.18:4827 , piece_id=BDXPRH2X2RMHWRN7TQSPWGZWC372GORJ3KDMHHUDO4XFJ73CQBHQ, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.125610159 (814h44m7.826555054s) [noise=true , address=145.79.227.6:4285 , piece_id=D5CCQKOJCXS7O3GOPDFIABFWC5Q6MXGKSSQ6GWROOBPFRPKTZ45A, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.126255859 (814h44m7.827200745s) [noise=true , address=77.90.31.13:4654 , piece_id=WY2HVJXW35GCO5FWD67SOUADYRPD7WXVQQS5CVXSUGWWDTB3KUJA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.126519425 (814h44m7.82746431s) [noise=true , address=77.90.31.16:4755 , piece_id=HO4YAWPFRTCSBPT3P6H4TX4OLGJTEVW2NFQN7XW5SJEGYGKCGNBQ, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.126758844 (814h44m7.827703729s) [noise=true , address=77.90.31.11:4574 , piece_id=JADCCMMF76TCKUWKTHM4KUDB4VE3EMYY3U7YADFFF6Y3UXYQFVOA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.127024382 (814h44m7.827969277s) [noise=true , address=77.90.31.10:4551 , piece_id=F3I5ZAOOJIIXGJX3TD7GFZSGJJPEPY3MZIM5FQJTUX5Q64CRW4AA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.127271145 (814h44m7.82821604s) [noise=true , address=45.91.116.12:4234 , piece_id=JNYC2CXQ5XULNLF3SH6YBKQIVIJTR3MFRCTAFBZJ4YOYHSTUICJQ, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.127536814 (814h44m7.828481709s) [noise=true , address=45.91.116.4:4956 , piece_id=PB4IY5DWE5FP6VHT37X2KOTNTSBLY6AENKHOY3AIS5LDMPPKXC7Q, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.127811709 (814h44m7.828756604s) [noise=true , address=45.91.116.10:4113 , piece_id=LVUPKF6DBSO277L6RDLWO44XINC2GSAC7XQO75ZSGXPMZZJDWKVA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.128129666 (814h44m7.829074561s) [noise=true , address=77.90.31.15:4715 , piece_id=EN2XBJRC2CXATGNMTXNSDAIY3VFTFP3WTUIKCLY5T3QAWC5Z6OZA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.128422255 (814h44m7.829367141s) [noise=true , address=45.91.116.6:4992 , piece_id=NU76RPQXISIZAO23XCALJNPIZFGCUDFM3CORT2SQ4YXQPGQ6EQDA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.128719172 (814h44m7.829664058s) [noise=true , address=45.91.116.9:4265 , piece_id=4BHJQLPLTGPQTW7B2J6SX7P66SWWQWXRPOO6KIWXNM7DRPD3DU5Q, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.129031188 (814h44m7.829976083s) [noise=true , address=145.79.227.4:4202 , piece_id=6ZT2PKP6XZBS7V52ZSYZVWDEGC6STDNU45TTB5IMMSZWMQF6AIIA, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.129995247 (814h44m7.830940132s) [noise=true , address=64.49.13.169:4714 , piece_id=ASHEOA3722YOMTLEX635TDFJM53VIOWHVCIS37CVL4KBDI5IOTNQ, seg_pos={0 0}, upload=716833]: Uploading piece...
Nov 30 09:57:41.218423178 (814h44m7.919368094s) [upload=716833]: Next returned nil segment
Nov 30 09:57:41.218440521 (814h44m7.919385435s) [upload=716833]: Waiting for error group managing segments...
Nov 30 09:57:41.227517704 (814h44m7.928462629s) [noise=true , address=45.91.116.9:4265 , piece_id=4BHJQLPLTGPQTW7B2J6SX7P66SWWQWXRPOO6KIWXNM7DRPD3DU5Q, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:41.228206046 (814h44m7.929150961s) [noise=true , address=45.91.116.4:4956 , piece_id=PB4IY5DWE5FP6VHT37X2KOTNTSBLY6AENKHOY3AIS5LDMPPKXC7Q, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:41.228350497 (814h44m7.929295412s) [noise=true , address=45.91.116.10:4113 , piece_id=LVUPKF6DBSO277L6RDLWO44XINC2GSAC7XQO75ZSGXPMZZJDWKVA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:41.229303305 (814h44m7.93024822s) [noise=true , address=145.79.227.6:4285 , piece_id=D5CCQKOJCXS7O3GOPDFIABFWC5Q6MXGKSSQ6GWROOBPFRPKTZ45A, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:41.229337128 (814h44m7.930282054s) [noise=true , address=45.91.116.12:4234 , piece_id=JNYC2CXQ5XULNLF3SH6YBKQIVIJTR3MFRCTAFBZJ4YOYHSTUICJQ, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:41.230159953 (814h44m7.931104867s) [noise=true , address=45.91.116.6:4992 , piece_id=NU76RPQXISIZAO23XCALJNPIZFGCUDFM3CORT2SQ4YXQPGQ6EQDA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:41.231047628 (814h44m7.931992544s) [noise=true , address=145.79.227.4:4202 , piece_id=6ZT2PKP6XZBS7V52ZSYZVWDEGC6STDNU45TTB5IMMSZWMQF6AIIA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:41.236903918 (814h44m7.937848833s) [noise=true , address=145.79.227.2:4124 , piece_id=2APJUPF5QSXVE2BCBDSXZVXG6LGOOBN6EGNGJTACE7UHDVRJE2TA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:41.682424860 (814h44m8.383369775s) [noise=true , address=64.49.13.137:4495 , piece_id=JXC26YQ2P3GI6JMDSZBFCFCH7KU2NHLTXXQ7TJ34QK4GSW45XKKQ, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:57:42.968954109 (814h44m9.669899005s) [noise=true , address=64.49.13.169:4714 , piece_id=ASHEOA3722YOMTLEX635TDFJM53VIOWHVCIS37CVL4KBDI5IOTNQ, seg_pos={0 0}, upload=716833]: Done uploading piece. err: <nil>
Nov 30 09:58:41.093075112 (814h45m7.794019998s) [noise=true , address=77.90.31.11:4574 , piece_id=JADCCMMF76TCKUWKTHM4KUDB4VE3EMYY3U7YADFFF6Y3UXYQFVOA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: rpc: dial tcp 77.90.31.11:4574: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:41.093161425 (814h45m7.79410631s) [noise=true , address=77.90.31.18:4827 , piece_id=BDXPRH2X2RMHWRN7TQSPWGZWC372GORJ3KDMHHUDO4XFJ73CQBHQ, seg_pos={0 0}, upload=716833]: Done uploading piece. err: ecclient: upload failed (node:127KQ2X6VXddEeFRMau7MVGwtTsUmFsm8hTYDR2A6B4A1oVKfyo, address:77.90.31.18:4827): protocol: rpc: dial tcp 77.90.31.18:4827: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:41.093186152 (814h45m7.794131036s) [noise=true , address=77.90.31.16:4755 , piece_id=HO4YAWPFRTCSBPT3P6H4TX4OLGJTEVW2NFQN7XW5SJEGYGKCGNBQ, seg_pos={0 0}, upload=716833]: Done uploading piece. err: ecclient: upload failed (node:1JsnHuhQhM4rWLTLEuSSHLNE24m7tSW9qMSf48j7yMrAVPWVyV, address:77.90.31.16:4755): protocol: rpc: dial tcp 77.90.31.16:4755: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:41.093267434 (814h45m7.794212319s) [noise=true , address=77.90.31.13:4654 , piece_id=WY2HVJXW35GCO5FWD67SOUADYRPD7WXVQQS5CVXSUGWWDTB3KUJA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: ecclient: upload failed (node:1227esgYd6VByyMSV6ZBcvYqx6PahMt7SUgvYzYmiWJavNXGza8, address:77.90.31.13:4654): protocol: rpc: dial tcp 77.90.31.13:4654: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:41.093275209 (814h45m7.794220104s) [noise=true , address=77.90.31.10:4551 , piece_id=F3I5ZAOOJIIXGJX3TD7GFZSGJJPEPY3MZIM5FQJTUX5Q64CRW4AA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: ecclient: upload failed (node:1EWyjRCErG9odC8xXPFaLXNWfsega2SLNbKHjbawahraGqDCwr, address:77.90.31.10:4551): protocol: rpc: dial tcp 77.90.31.10:4551: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:41.093357934 (814h45m7.794302819s) [noise=true , address=77.90.31.15:4715 , piece_id=EN2XBJRC2CXATGNMTXNSDAIY3VFTFP3WTUIKCLY5T3QAWC5Z6OZA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: ecclient: upload failed (node:12WXsguvsox5t3gJTmYgXxSvS9Wha4DSSzhfadhoASU4see1FCs, address:77.90.31.15:4715): protocol: rpc: dial tcp 77.90.31.15:4715: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:41.093384243 (814h45m7.794329138s) [noise=true , address=77.90.31.14:4665 , piece_id=V6E4QAFOP33HX2GH5UGMJNYAIHFORO5BXNL7JTITMLQ43WEA7R3A, seg_pos={0 0}, upload=716833]: Done uploading piece. err: ecclient: upload failed (node:12W4n6uf6azQk1QgeR8Xw7UkpnZm19zdyFKwojCsqZt2GFHrpce, address:77.90.31.14:4665): protocol: write tcp 5.161.89.252:33216->77.90.31.14:4665: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:41.093433164 (814h45m7.79437805s) [noise=true , address=77.90.31.21:4899 , piece_id=YRENXKSRXUOQS7O7A54DJVPYO4XFKKHH76LVPHLIURRMICUC7CYA, seg_pos={0 0}, upload=716833]: Done uploading piece. err: ecclient: upload failed (node:12UjFkFn2UEvr57LDorcijDCvkBkkKitgZEFx6Yqfk5gMStQvJG, address:77.90.31.21:4899): protocol: write tcp 5.161.89.252:19594->77.90.31.21:4899: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:41.093450177 (814h45m7.794395062s) [seg_pos={0 0}, upload=716833]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
Nov 30 09:58:41.108144572 (814h45m7.809089457s) [upload=716833]: Done uploading segments...
Nov 30 09:58:41.108157476 (814h45m7.809102371s) [upload=716833]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 1m0.017002617s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.020062744s
FAIL (2025-11-30 09:58:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 30 09:57:54.338190879 (814h44m21.039135763s) [upload=716835]: Starting upload
Nov 30 09:57:54.338244719 (814h44m21.039189614s) [upload=716835]: Done starting upload
Nov 30 09:57:54.338384973 (814h44m21.039329868s) [upload=716835]: Uploading segments...
Nov 30 09:57:54.338487656 (814h44m21.039432541s) [upload=716835]: Got next segment. Inline: false
Nov 30 09:57:54.338526589 (814h44m21.039471494s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 30 09:57:54.338559952 (814h44m21.039504877s) [upload=716835]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 30 09:57:54.361097355 (814h44m21.06204224s) [seg_pos={0 0}, upload=716835]: Begin upload segment...
Nov 30 09:57:54.361157588 (814h44m21.062102462s) [seg_pos={0 0}, upload=716835]: Done begin upload segment.
Nov 30 09:57:54.361186011 (814h44m21.062130916s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361205928 (814h44m21.062150823s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361626347 (814h44m21.062571232s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.361930398 (814h44m21.062875293s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362176009 (814h44m21.063120904s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362428763 (814h44m21.063373648s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362677220 (814h44m21.063622105s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.362933340 (814h44m21.063878225s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363160577 (814h44m21.064105472s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.363400687 (814h44m21.064345582s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364097124 (814h44m21.065042009s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364511481 (814h44m21.065456386s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.364999117 (814h44m21.065944012s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365062575 (814h44m21.06600747s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365334095 (814h44m21.06627898s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.365625122 (814h44m21.066570006s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Uploading piece...
Nov 30 09:57:54.437461809 (814h44m21.138406694s) [upload=716835]: Next returned nil segment
Nov 30 09:57:54.437472619 (814h44m21.138417504s) [upload=716835]: Waiting for error group managing segments...
Nov 30 09:57:54.446112432 (814h44m21.147057317s) [noise=true , address=45.91.116.8:4045 , piece_id=I5IJZ7VBOWMXUAPBUJP5YYMGX3GSS4SOWZUWOOMM7XAZRTM7AUMQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.446603695 (814h44m21.147548609s) [noise=true , address=45.91.116.10:4081 , piece_id=WUQEQDP2VS2EHLLO7RGS462V4FKUOVXGLZRX25ZVVTUT3HGOIA5Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447195905 (814h44m21.1481408s) [noise=true , address=145.79.227.39:4015 , piece_id=KAB5SQBQPWGHKGEYMTBD2VTCAURBYEZ2LT7B4XESCIUG5PMCHJQA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447430665 (814h44m21.148375551s) [noise=true , address=145.79.227.42:4121 , piece_id=5OICIQHCUXLBKC7VOHNSNF7FM3EWDLM7QZG74EGUY7OSWDGTCLKQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447592850 (814h44m21.148537736s) [noise=true , address=145.79.227.24:4582 , piece_id=RAZUHVJ7XPITPIENN46YV5KO2LXIXHW4L7ZRMIURYJXDT2SYJWWA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.447971461 (814h44m21.148916346s) [noise=true , address=145.79.227.26:4655 , piece_id=CE4V4WNY6N6XLC6FCFLVENKPMEXBZJWCFJKCJ2N7LQOOAUQLU2LQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448046792 (814h44m21.148991688s) [noise=true , address=145.79.227.4:4186 , piece_id=NKYJRESISGAEEOBIIKHFBHMRXB74WRBPJORJJH7WQG4APO7SD7LA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448347225 (814h44m21.149292111s) [noise=true , address=45.91.116.11:4119 , piece_id=SJ7S6JCLNY3MID3DLIXWRM7RF2GNSGXXWSMMMMY62JB7O42UA6EQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448489934 (814h44m21.149434819s) [noise=true , address=145.79.227.28:4726 , piece_id=RXBLNITDMQMXDL5YCDYE5QYZOS6CUO35JHXYPBRIY7SJ4M7ESXTA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448666054 (814h44m21.14961095s) [noise=true , address=45.91.116.5:4299 , piece_id=WWCR5BCEITUBHUHYQQWZ3XUJCVYZ2PRQYBLPDQSPZLP732M33MDA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.448904582 (814h44m21.149849467s) [noise=true , address=145.79.227.30:4761 , piece_id=CGG3PQB52EIVJH7IMVQTWQ2APIJ5SATHWQIJDMRYSW4ULIKYHZLQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:57:54.449463692 (814h44m21.150408587s) [noise=true , address=145.79.227.38:4977 , piece_id=VV64FHNNAX7ZX6ITWO7UBLWQEAK4YBRKPZHDEQ27FY4G3VSUXOVQ, seg_pos={0 0}, upload=716835]: Done uploading piece. err: <nil>
Nov 30 09:58:24.327129404 (814h44m51.028074289s) [noise=true , address=77.90.31.10:4520 , piece_id=IS2R6IMSFXSVW7QBYVEZA36VU3QAQQ77RH6ZEFXRD5B2ZPT7GG6Q, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: write tcp 5.161.89.252:27704->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327148590 (814h44m51.028093476s) [noise=true , address=77.90.31.19:4842 , piece_id=XJT5ZQA3UKQ64FMC2K2JSHADLHUUWIGINSZ5S6CMEWU6YPPIGYBA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327211238 (814h44m51.028156123s) [noise=true , address=77.90.31.16:4734 , piece_id=Y7ZFZGGBGXMDKOIKVA5IDOEKWAO6O5UW4LUBFWQYUC74NZ5DY7CA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 5.161.89.252:45892->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327233088 (814h44m51.028177984s) [noise=true , address=77.90.31.15:4699 , piece_id=VVNU63DNNIDA2SL3KWGRB4LETBZ2F5MPSFIRKRHUMWNUDOOCB5IA, seg_pos={0 0}, upload=716835]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: write tcp 5.161.89.252:21530->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 30 09:58:24.327250742 (814h44m51.028195637s) [seg_pos={0 0}, upload=716835]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 30 09:58:24.345660629 (814h44m51.046605515s) [upload=716835]: Done uploading segments...
Nov 30 09:58:24.345673454 (814h44m51.046618349s) [upload=716835]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.0200927s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
310.86964ms
FAIL (2025-11-30 09:58:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 310.88495ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
5.000598884s
FAIL (2025-11-30 09:58:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000614574s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
334.957311ms
FAIL (2025-11-30 09:57:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 334.975706ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001767928s
FAIL (2025-11-30 09:57:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001788345s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
5.000649139s
FAIL (2025-11-30 09:57:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000668294s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
118.004065ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:17:25 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 118.025694ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
124.408429ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:17:04 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 124.464725ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
120.029896ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:15:57 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 120.052749ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
152.809972ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:15:30 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 152.826333ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
124.804482ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:14:53 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 124.816757ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
121.570896ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:14:05 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 121.590864ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
121.38789ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:13:05 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 121.412075ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
125.072476ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:12:05 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 125.091592ms
edge
linksharing
bgp
eu1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.eu1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
133.686314ms remote:
"tcp:136.0.77.2:443" target:
https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt
FAIL (2025-11-30 01:11:00 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwhnbh22a7huxgm5mpnu45kaahuq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 133.737029ms
edge
linksharing
bgp
eu1
|