feat: 全量標註的檢查、重標與合併工具 (#5)
verify_batches.py 逐批比對 id;make_relabel.py 挑出需重標的 none; merge_labels.py 支援多個目錄覆蓋與 confidence 縮寫;worry 加入情緒白名單。 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5.5
parent
c85e00a56e
commit
ea6b9ca7ef
@@ -3,7 +3,7 @@
|
||||
|
||||
用法:
|
||||
make_batches.py <dialogues.jsonl> <輸出目錄> --pilot N # 試跑:分層抽樣 N 句
|
||||
make_batches.py <dialogues.jsonl> <輸出目錄> --all # 全量:規則未判定或中信心的對白
|
||||
make_batches.py <dialogues.jsonl> <輸出目錄> --all # 全量:所有對白(規則高信心也只有約八成準,所以一併標註)
|
||||
|
||||
每批一個 jsonl,一行一句,附前後各 2 句對白當輪流發言的線索。
|
||||
批次檔不含規則判定的說話者,讓標註獨立,之後才能拿來和規則比對。
|
||||
@@ -57,7 +57,7 @@ def main():
|
||||
ap.error("--pilot N 與 --all 擇一")
|
||||
recs = [json.loads(line) for line in Path(a.dialogues).read_text(encoding="utf-8").splitlines()]
|
||||
nb = with_neighbors(recs)
|
||||
chosen = pilot_sample(recs, a.pilot, a.seed) if a.pilot else [r for r in recs if r["confidence"] != "high"]
|
||||
chosen = pilot_sample(recs, a.pilot, a.seed) if a.pilot else recs
|
||||
out = Path(a.outdir)
|
||||
out.mkdir(parents=True, exist_ok=True)
|
||||
for i in range(0, len(chosen), a.batch_size):
|
||||
|
||||
Reference in New Issue
Block a user