Starting fold 1/5
0: learn: 0.6720091 total: 303ms remaining: 30s
50: learn: 0.1853333 total: 12.5s remaining: 12s
99: learn: 0.0980875 total: 24.3s remaining: 0us
Fold 1/5 completed
Starting fold 2/5
0: learn: 0.6691159 total: 282ms remaining: 27.9s
50: learn: 0.2106832 total: 12.4s remaining: 11.9s
99: learn: 0.1109963 total: 24.5s remaining: 0us
Fold 2/5 completed
Starting fold 3/5
0: learn: 0.6727001 total: 286ms remaining: 28.4s
50: learn: 0.2015422 total: 12.6s remaining: 12.1s
99: learn: 0.1078658 total: 24.6s remaining: 0us
Fold 3/5 completed
Starting fold 4/5
0: learn: 0.6673562 total: 296ms remaining: 29.3s
50: learn: 0.1926436 total: 12.6s remaining: 12.1s
99: learn: 0.1034305 total: 24.8s remaining: 0us
Fold 4/5 completed
Starting fold 5/5
0: learn: 0.6729438 total: 298ms remaining: 29.5s
50: learn: 0.1964025 total: 13.1s remaining: 12.6s
99: learn: 0.1022904 total: 25.1s remaining: 0us
Fold 5/5 completed
CatBoost, threshold=0.5
precision recall f1-score support
in 0.948 0.936 0.942 549
out 0.722 0.765 0.743 119
accuracy 0.906 668
macro avg 0.835 0.850 0.843 668
weighted avg 0.908 0.906 0.907 668
Confusion matrix [in, out]:
[[514 35]
[ 28 91]]
CatBoost, safe threshold=0.697862, target recall_in=0.99
precision recall f1-score support
in 0.920 0.991 0.954 549
out 0.935 0.605 0.735 119
accuracy 0.922 668
macro avg 0.928 0.798 0.845 668
weighted avg 0.923 0.922 0.915 668
Confusion matrix [in, out]:
[[544 5]
[ 47 72]]
0: learn: 0.6709096 total: 294ms remaining: 29.1s
50: learn: 0.2103099 total: 12.6s remaining: 12.1s
99: learn: 0.1217924 total: 24.5s remaining: 0us
{'model_type': 'CatBoostClassifier',
'embedding_model': 'frida',
'embedding_base_url': 'https://api.ai.beeline.ru/api/v3',
'embedding_prefix': 'categorize_topic: ',
'train_size': 668,
'expert_examples': 165,
'synthetic_examples': 503,
'classes': ['in', 'out'],
'parameters': {'iterations': 100,
'depth': 5,
'learning_rate': 0.05,
'l2_leaf_reg': 10,
'loss_function': 'Logloss',
'auto_class_weights': 'Balanced',
'random_seed': 42,
'thread_count': -1,
'verbose': 50,
'allow_writing_files': False},
'default_threshold': 0.5,
'recommended_out_threshold': 0.6978621146118229,
'target_recall_in': 0.99,
'default_f1_in': 0.9422548120989918,
'default_f1_out': 0.7428571428571429,
'safe_f1_in': 0.9543859649122807,
'safe_f1_out': 0.7346938775510204}