Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
416.887484ms
FAIL (2025-11-28 22:46:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 416.900759ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X...............................X.....................] duration:
369.878217ms
FAIL (2025-11-28 22:45:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 369.901251ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
483.632025ms
FAIL (2025-11-28 22:45:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 483.652043ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.714081527s
FAIL (2025-11-28 22:45:40 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.714106173s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X...............................X....................] duration:
394.208257ms
FAIL (2025-11-28 22:45:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 394.226281ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
521.733619ms
FAIL (2025-11-28 22:45:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 521.761522ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
472.679689ms
FAIL (2025-11-28 22:44:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 472.701871ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X...............................X...................] duration:
369.819906ms
FAIL (2025-11-28 22:44:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 369.855713ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.720291376s
FAIL (2025-11-28 22:44:43 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.720323496s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
485.941085ms
FAIL (2025-11-28 22:44:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 485.961092ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........X...............................X..................] duration:
347.669712ms
FAIL (2025-11-28 22:44:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 347.686745ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
504.617825ms
FAIL (2025-11-28 22:44:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 504.666778ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
497.613302ms
FAIL (2025-11-28 22:44:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 497.660891ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.68066541s
FAIL (2025-11-28 22:43:52 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.680685849s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
457.481951ms
FAIL (2025-11-28 22:43:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 457.505134ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X...............................X.................] duration:
399.096332ms
FAIL (2025-11-28 22:43:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 399.13246ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X...............................X................] duration:
401.71414ms
FAIL (2025-11-28 22:43:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 401.738245ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.776397428s
FAIL (2025-11-28 22:43:15 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.776432976s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
486.282045ms
FAIL (2025-11-28 22:43:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 486.303264ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
508.54141ms
FAIL (2025-11-28 22:43:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 508.590142ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
499.313029ms
FAIL (2025-11-28 22:42:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 499.332544ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
485.454179ms
FAIL (2025-11-28 22:42:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 485.478224ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X...............................X...............] duration:
356.747424ms
FAIL (2025-11-28 22:42:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 356.765298ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X...............................X..............] duration:
408.723978ms
FAIL (2025-11-28 22:41:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 408.744457ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
519.042734ms
FAIL (2025-11-28 22:41:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 519.206051ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
465.900198ms
FAIL (2025-11-28 22:41:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 465.921316ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
513.203284ms
FAIL (2025-11-28 22:41:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 513.2273ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X...............................X.............] duration:
412.086991ms
FAIL (2025-11-28 22:40:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 412.184574ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
499.094499ms
FAIL (2025-11-28 22:40:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 499.103556ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
411.618463ms
FAIL (2025-11-28 22:40:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 411.6433ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X...............................X............] duration:
389.046276ms
FAIL (2025-11-28 22:40:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 389.065743ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................X...............................X...........] duration:
362.121592ms
FAIL (2025-11-28 22:39:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 362.165594ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
420.295734ms
FAIL (2025-11-28 22:39:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 420.319278ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
732.273862ms
FAIL (2025-11-28 22:39:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 732.292867ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................X...............................X..........] duration:
417.461979ms
FAIL (2025-11-28 22:39:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 417.479772ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
492.889124ms
FAIL (2025-11-28 22:38:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 493.026641ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
436.194655ms
FAIL (2025-11-28 22:38:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 436.222367ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X...............................X.........] duration:
399.401189ms
FAIL (2025-11-28 22:38:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 399.421877ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
502.309306ms
FAIL (2025-11-28 22:37:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 502.331206ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
461.462814ms
FAIL (2025-11-28 22:37:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 461.479386ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
443.365391ms
FAIL (2025-11-28 22:37:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 443.399295ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X...............................X........] duration:
394.942381ms
FAIL (2025-11-28 22:37:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 395.096079ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
476.520155ms
FAIL (2025-11-28 22:37:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 476.540903ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
488.780567ms
FAIL (2025-11-28 22:37:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 488.801396ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
472.378565ms
FAIL (2025-11-28 22:36:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 472.396198ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X...............................X.......] duration:
344.534457ms
FAIL (2025-11-28 22:36:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 344.549334ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
509.43544ms
FAIL (2025-11-28 22:36:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 509.464114ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
458.18771ms
FAIL (2025-11-28 22:36:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 458.226893ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
541.825513ms
FAIL (2025-11-28 22:35:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 13)
======LOGS======
685337]: Got next segment. Inline: false
Nov 28 22:07:45.755017846 (778h54m12.455962761s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 28 22:07:45.755023667 (778h54m12.455968582s) [upload=685337]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 28 22:07:45.778708895 (778h54m12.47965379s) [seg_pos={0 0}, upload=685337]: Begin upload segment...
Nov 28 22:07:45.778745854 (778h54m12.479690749s) [seg_pos={0 0}, upload=685337]: Done begin upload segment.
Nov 28 22:07:45.778765340 (778h54m12.479710225s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.778775700 (778h54m12.479720595s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779062417 (778h54m12.480007303s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779307377 (778h54m12.480252272s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779575400 (778h54m12.480520295s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.779823907 (778h54m12.480768802s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780073154 (778h54m12.481018039s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780325457 (778h54m12.481270352s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780580225 (778h54m12.48152512s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.780820516 (778h54m12.481765401s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781491926 (778h54m12.482436821s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.781855017 (778h54m12.482799912s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782200475 (778h54m12.48314537s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782505077 (778h54m12.483449972s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.782688932 (778h54m12.483633827s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783017358 (778h54m12.483962253s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783223365 (778h54m12.48416826s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.783599079 (778h54m12.484543964s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Uploading piece...
Nov 28 22:07:45.846053485 (778h54m12.54699838s) [upload=685337]: Next returned nil segment
Nov 28 22:07:45.846070156 (778h54m12.547015141s) [upload=685337]: Waiting for error group managing segments...
Nov 28 22:07:45.854930911 (778h54m12.555875815s) [noise=true , address=145.79.227.18:4038 , piece_id=ZZLRO4YRCWTEDUWH6NOO5MXGK4XYOB7HIQZ6OM7N2H55RABW62JA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855091713 (778h54m12.556036617s) [noise=true , address=77.90.31.17:4778 , piece_id=M66OFE6QCOKUL6M4T6WSEMVKG7BQ3C3DTGNHOB6LDMOUYWJOP3PA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.855902954 (778h54m12.556847859s) [noise=true , address=145.79.227.33:4878 , piece_id=HNWAJWLLCSRYTRSHBTEDOJWPQEYWQREX2DX3KMIDIATTE6PHBWZQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857845339 (778h54m12.558790243s) [noise=true , address=77.90.31.15:4719 , piece_id=5XS752NHATLM2JJWZDQFYPQVIBIBCLMMIY636SL4W2IBD2F6P3GQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.857955355 (778h54m12.55890026s) [noise=true , address=145.79.227.44:4206 , piece_id=2XPEXIA3AJIEUSNJLQNJP7WQZKOYO3TC2F6ESOAKOE5WQT2XEZWQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.858253525 (778h54m12.559198429s) [noise=true , address=77.90.31.18:4810 , piece_id=VT5BQGHINSMIRB66KCPQYHBUNIHHWTR6HV25Y42S4KUUK6TUPP3Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864341055 (778h54m12.565285971s) [noise=true , address=77.90.31.21:4887 , piece_id=SECEKOXHHUQX2SOESFJL3VHDI676CZMQWDITLFXHBGU6IDGRT4BQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.864818060 (778h54m12.565762965s) [noise=true , address=145.79.227.12:4796 , piece_id=PEEPORM2VQQOC62VUMCGA4KIOVOV2CGB4L54X4RXNIUPUADQXJMA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.868107551 (778h54m12.569052467s) [noise=true , address=145.79.227.9:4389 , piece_id=XEMIPG7O4VK77K5R3R3HH4NE5AKATF2NN2L6QCI47P4AEHDLEV5A, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.875789043 (778h54m12.576733959s) [noise=true , address=77.90.31.12:4617 , piece_id=BAKZHAB47DY5EWA5W6GQK3XZPKCLKN6OK4RUGMM2RL2KMV5SJP6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:45.882872613 (778h54m12.583817529s) [noise=true , address=145.79.227.16:4938 , piece_id=KYSYENOVG65C4MVJ5GFFJCEAYY6QPGNUMZFKLBYPT6A255MIHSSQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:46.354270809 (778h54m13.055215704s) [noise=true , address=64.49.13.170:4722 , piece_id=JQZIRS4W4NYMC4S3BKV7II53RWTWBCFGJD7GURBZVIOJ46NEEIRA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:07:47.317446642 (778h54m14.018391537s) [noise=true , address=64.49.13.169:4710 , piece_id=RHYEQNIKAYKM3TCHHXXEWPZT3FDQ4JIJKILWK43G57MIZ36E4CBA, seg_pos={0 0}, upload=685337]: Done uploading piece. err: <nil>
Nov 28 22:08:45.741611849 (778h55m12.442556744s) [noise=true , address=45.91.116.14:4220 , piece_id=DPOFFPVE6OREBRPYCGL5IG3CHRJJSCTAG55CQKIW6U5CCGHPEHHQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:12ZH3hS6xJp29U2RWAN9p8kxNdkSUQr1E58HzcFei48fUbNzFvW, address:45.91.116.14:4220): protocol: rpc: dial tcp 45.91.116.14:4220: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741625925 (778h55m12.44257082s) [noise=true , address=45.91.116.7:4033 , piece_id=JXOQUCARCQ47W3UIQBERGIYPYBG2NZEWZ7OE2BC5NVIBPAPSCE2Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Mg5mxUjW56quDidSgA788D7ByfAzvsEuWJqwtzY4bLyxRn3mu, address:45.91.116.7:4033): protocol: write tcp 5.161.89.252:39072->45.91.116.7:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741709111 (778h55m12.442654006s) [noise=true , address=45.91.116.7:4025 , piece_id=QVQD6ZTBWZCZ4THMPRUP5IA6RCM7KG33LU5QTPVHEB6WPU2AKO5Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1WnhUZYDuxiqz9Uji8mhnnHng724Qk9UEJEXyuHLEhbfJymMZj, address:45.91.116.7:4025): protocol: read tcp 5.161.89.252:12040->45.91.116.7:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741732926 (778h55m12.442677821s) [noise=true , address=45.91.116.13:4159 , piece_id=R3Z55SUVTEULT2XGGOOGKJLDL4QIXT4R3HODMSMAH5OSBFISUKVQ, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:128V9AYe3XZKQ9o2cbdeSNYoK55MHFJ4q1KbhjGXWzKrv17Hv3t, address:45.91.116.13:4159): protocol: rpc: dial tcp 45.91.116.13:4159: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741776548 (778h55m12.442721432s) [noise=true , address=45.91.116.11:4122 , piece_id=BOSR3QZSILW2GW4P3DRLQCG332ODFY44DATMHO3MESSEGESJXW6Q, seg_pos={0 0}, upload=685337]: Done uploading piece. err: ecclient: upload failed (node:1Kg43gDX5Tybqk93Xg9F1rVzGjbDGJYrrkDUUK4Mepdu2iGYJm, address:45.91.116.11:4122): protocol: read tcp 5.161.89.252:8108->45.91.116.11:4122: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 28 22:08:45.741795153 (778h55m12.442740048s) [seg_pos={0 0}, upload=685337]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil>
Nov 28 22:08:45.761066499 (778h55m12.462011383s) [upload=685337]: Done uploading segments...
Nov 28 22:08:45.761081336 (778h55m12.462026221s) [upload=685337]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13)
Duration: 541.850079ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................X...............................X......] duration:
366.538306ms
FAIL (2025-11-28 22:35:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 366.680986ms
edge
gateway
us-select-1
|