Загрузка данных


Starting fold 1/5
0:	learn: 0.6720091	total: 303ms	remaining: 30s
50:	learn: 0.1853333	total: 12.5s	remaining: 12s
99:	learn: 0.0980875	total: 24.3s	remaining: 0us
Fold 1/5 completed

Starting fold 2/5
0:	learn: 0.6691159	total: 282ms	remaining: 27.9s
50:	learn: 0.2106832	total: 12.4s	remaining: 11.9s
99:	learn: 0.1109963	total: 24.5s	remaining: 0us
Fold 2/5 completed

Starting fold 3/5
0:	learn: 0.6727001	total: 286ms	remaining: 28.4s
50:	learn: 0.2015422	total: 12.6s	remaining: 12.1s
99:	learn: 0.1078658	total: 24.6s	remaining: 0us
Fold 3/5 completed

Starting fold 4/5
0:	learn: 0.6673562	total: 296ms	remaining: 29.3s
50:	learn: 0.1926436	total: 12.6s	remaining: 12.1s
99:	learn: 0.1034305	total: 24.8s	remaining: 0us
Fold 4/5 completed

Starting fold 5/5
0:	learn: 0.6729438	total: 298ms	remaining: 29.5s
50:	learn: 0.1964025	total: 13.1s	remaining: 12.6s
99:	learn: 0.1022904	total: 25.1s	remaining: 0us
Fold 5/5 completed

CatBoost, threshold=0.5
              precision    recall  f1-score   support

          in      0.948     0.936     0.942       549
         out      0.722     0.765     0.743       119

    accuracy                          0.906       668
   macro avg      0.835     0.850     0.843       668
weighted avg      0.908     0.906     0.907       668

Confusion matrix [in, out]:
[[514  35]
 [ 28  91]]

CatBoost, safe threshold=0.697862, target recall_in=0.99
              precision    recall  f1-score   support

          in      0.920     0.991     0.954       549
         out      0.935     0.605     0.735       119

    accuracy                          0.922       668
   macro avg      0.928     0.798     0.845       668
weighted avg      0.923     0.922     0.915       668

Confusion matrix [in, out]:
[[544   5]
 [ 47  72]]
0:	learn: 0.6709096	total: 294ms	remaining: 29.1s
50:	learn: 0.2103099	total: 12.6s	remaining: 12.1s
99:	learn: 0.1217924	total: 24.5s	remaining: 0us
{'model_type': 'CatBoostClassifier',
 'embedding_model': 'frida',
 'embedding_base_url': 'https://api.ai.beeline.ru/api/v3',
 'embedding_prefix': 'categorize_topic: ',
 'train_size': 668,
 'expert_examples': 165,
 'synthetic_examples': 503,
 'classes': ['in', 'out'],
 'parameters': {'iterations': 100,
  'depth': 5,
  'learning_rate': 0.05,
  'l2_leaf_reg': 10,
  'loss_function': 'Logloss',
  'auto_class_weights': 'Balanced',
  'random_seed': 42,
  'thread_count': -1,
  'verbose': 50,
  'allow_writing_files': False},
 'default_threshold': 0.5,
 'recommended_out_threshold': 0.6978621146118229,
 'target_recall_in': 0.99,
 'default_f1_in': 0.9422548120989918,
 'default_f1_out': 0.7428571428571429,
 'safe_f1_in': 0.9543859649122807,
 'safe_f1_out': 0.7346938775510204}