mirror of https://github.com/microsoft/autogen.git
6 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
EgorKraevTransferwise
|
5245efbd2c
|
Factor out time series-related functionality into a time series Task object (#989)
* Refactor into automl subpackage Moved some of the packages into an automl subpackage to tidy before the task-based refactor. This is in response to discussions with the group and a comment on the first task-based PR. Only changes here are moving subpackages and modules into the new automl, fixing imports to work with this structure and fixing some dependencies in setup.py. * Fix doc building post automl subpackage refactor * Fix broken links in website post automl subpackage refactor * Fix broken links in website post automl subpackage refactor * Remove vw from test deps as this is breaking the build * Move default back to the top-level I'd moved this to automl as that's where it's used internally, but had missed that this is actually part of the public interface so makes sense to live where it was. * Re-add top level modules with deprecation warnings flaml.data, flaml.ml and flaml.model are re-added to the top level, being re-exported from flaml.automl for backwards compatability. Adding a deprecation warning so that we can have a planned removal later. * Fix model.py line-endings * WIP * WIP - Notes below Got to the point where the methods from AutoML are pulled to GenericTask. Started removing private markers and removing the passing of automl to these methods. Done with decide_split_type, started on prepare_data. Need to do the others after * Re-add generic_task * Most of the merge done, test_forecast_automl fit succeeds, fails at predict() * Remaining fixes - test_forecast.py passes * Comment out holidays-related code as it's not currently used * Further holidays cleanup * Fix imports in a test * tidy up validate_data in time series task * Test fixes * Fix tests: add Task.__str__ * Fix tests: test for ray.ObjectRef * Hotwire TS_Sklearn wrapper to fix test fail * Attempt at test fix * Fix test where val_pred_y is a list * Attempt to fix remaining tests * Push to retrigger tests * Push to retrigger tests * Push to retrigger tests * Push to retrigger tests * Remove plots from automl/test_forecast * Remove unused data size field from Task * Fix import for CLASSIFICATION in notebook * Monkey patch TFT to avoid plotting, to fix tests on MacOS * Monkey patch TFT to avoid plotting v2, to fix tests on MacOS * Monkey patch TFT to avoid plotting v2, to fix tests on MacOS * Fix circular import * remove redundant code in task.py post-merge * Fix test: set svd_solver="full" in PCA * Update flaml/automl/data.py Co-authored-by: Chi Wang <wang.chi@microsoft.com> * Fix review comments * Fix task -> str in custom learner constructor * Remove unused CLASSIFICATION imports * Hotwire TS_Sklearn wrapper to fix test fail by setting optimizer_for_horizon == False * Revert changes to the automl_classification and pin FLAML version * Fix imports in reverted notebook * Fix FLAML version in automl notebooks * Fix ml.py line endings * Fix CLASSIFICATION task import in automl_classification notebook * Uncomment pip install in notebook and revert import Not convinced this will work because of installing an older version of the package into the environment in which we're running the tests, but let's see. * Revert c6a5dd1a0 * Fix get_classification_objective import in suggest.py * Remove hcrystallball docs reference in TS_Sklearn * Merge markharley:extract-task-class-from-automl into this * Fix import, remove smooth.py * Fix dependencies to fix TFT fail on Windows Python 3.8 and 3.9 * Add tensorboardX dependency to fix TFT fail on Windows Python 3.8 and 3.9 * Set pytorch-lightning==1.9.0 to fix TFT fail on Windows Python 3.8 and 3.9 * Set pytorch-lightning==1.9.0 to fix TFT fail on Windows Python 3.8 and 3.9 * Disable PCA reduction of lagged features for now, to fix svd convervence fail * Merge flaml/main into time_series_task * Attempt to fix formatting * Attempt to fix formatting * tentatively implement holt-winters-no covariates * fix forecast method, clean class * checking external regressors too * update test forecast * remove duplicated test file, re-add sarimax, search space cleanup * Update flaml/automl/model.py removed links. Most important one probably was: https://robjhyndman.com/hyndsight/ets-regressors/ Co-authored-by: Chi Wang <wang.chi@microsoft.com> * prevent short series * add docs * First attempt at merging Holt-Winters * Linter fix * Add holt-winters to TimeSeriesTask.estimators * Fix spark test fail * Attempt to fix another spark test fail * Attempt to fix another spark test fail * Change Black max line length to 127 * Change Black max line length to 120 * Add logging for ARIMA params, clean up time series models inheritance * Add more logging for missing ARIMA params * Remove a meaningless test causing a fail, add stricter check on ARIMA params * Fix a bug in HoltWinters * A pointless change to hopefully trigger the on and off KeyError in ARIMA.fit() * Fix formatting * Attempt to fix formatting * Attempt to fix formatting * Attempt to fix formatting * Attempt to fix formatting * Add type annotations to _train_with_config() in state.py * Add type annotations to prepare_sample_train_data() in state.py * Add docstring for time_col argument of AutoML.fit() * Address @sonichi's comments on PR * Fix formatting * Fix formatting * Reduce test time budget * Reduce test time budget * Increase time budget for the test to pass * Remove redundant imports * Remove more redundant imports * Minor fixes of points raised by Qingyun * Try to fix pandas import fail * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Try to fix pandas import fail, again * Formatting fixes * More formatting fixes * Added test that loops over TS models to ensure coverage * Fix formatting issues * Fix more formatting issues * Fix random fail in check * Put back in tests for ARIMA predict without fit * Put back in tests for lgbm * Update test/test_model.py cover dedup * Match target length to X length in missing test --------- Co-authored-by: Mark Harley <mark.harley@transferwise.com> Co-authored-by: Mark Harley <mharley.code@gmail.com> Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu> Co-authored-by: Chi Wang <wang.chi@microsoft.com> Co-authored-by: Andrea W <a.ruggerini@ammagamma.com> Co-authored-by: Andrea Ruggerini <nescio.adv@gmail.com> Co-authored-by: Egor Kraev <Egor.Kraev@tw.com> Co-authored-by: Li Jiang <bnujli@gmail.com> |
|
|
Chi Wang
|
a0b318b12e
|
create an automl option to remove unnecessary dependency for autogen and tune (#1007)
* version update post release v1.2.2 * automl option * import pandas * remove automl.utils * default * test * type hint and version update * dependency update * link to open in colab * use packging.version to close #725 --------- Co-authored-by: Li Jiang <lijiang1@microsoft.com> Co-authored-by: Li Jiang <bnujli@gmail.com> |
|
|
Jirka Borovec
|
a701cd82f8
|
set black with 120 line length (#975)
* set black with 120 line length * apply pre-commit * apply black |
|
|
Andrea Ruggerini
|
7f9402b8fd
|
Add Holt-Winters exponential smoothing (#962)
* tentatively implement holt-winters-no covariates * fix forecast method, clean class * checking external regressors too * update test forecast * remove duplicated test file, re-add sarimax, search space cleanup * Update flaml/automl/model.py removed links. Most important one probably was: https://robjhyndman.com/hyndsight/ets-regressors/ Co-authored-by: Chi Wang <wang.chi@microsoft.com> * prevent short series * add docs --------- Co-authored-by: Andrea W <a.ruggerini@ammagamma.com> Co-authored-by: Chi Wang <wang.chi@microsoft.com> |
|
|
Li Jiang
|
50334f2c52
|
Support spark dataframe as input dataset and spark models as estimators (#934)
* add basic support to Spark dataframe add support to SynapseML LightGBM model update to pyspark>=3.2.0 to leverage pandas_on_Spark API * clean code, add TODOs * add sample_train_data for pyspark.pandas dataframe, fix bugs * improve some functions, fix bugs * fix dict change size during iteration * update model predict * update LightGBM model, update test * update SynapseML LightGBM params * update synapseML and tests * update TODOs * Added support to roc_auc for spark models * Added support to score of spark estimator * Added test for automl score of spark estimator * Added cv support to pyspark.pandas dataframe * Update test, fix bugs * Added tests * Updated docs, tests, added a notebook * Fix bugs in non-spark env * Fix bugs and improve tests * Fix uninstall pyspark * Fix tests error * Fix java.lang.OutOfMemoryError: Java heap space * Fix test_performance * Update test_sparkml to test_0sparkml to use the expected spark conf * Remove unnecessary widgets in notebook * Fix iloc java.lang.StackOverflowError * fix pre-commit * Added params check for spark dataframes * Refactor code for train_test_split to a function * Update train_test_split_pyspark * Refactor if-else, remove unnecessary code * Remove y from predict, remove mem control from n_iter compute * Update workflow * Improve _split_pyspark * Fix test failure of too short training time * Fix typos, improve docstrings * Fix index errors of pandas_on_spark, add spark loss metric * Fix typo of ndcgAtK * Update NDCG metrics and tests * Remove unuseful logger * Use cache and count to ensure consistent indexes * refactor for merge maain * fix errors of refactor * Updated SparkLightGBMEstimator and cache * Updated config2params * Remove unused import * Fix unknown parameters * Update default_estimator_list * Add unit tests for spark metrics |
|
|
Mark Harley
|
27b2712016
|
Extract task class from automl (#857)
* Refactor into automl subpackage Moved some of the packages into an automl subpackage to tidy before the task-based refactor. This is in response to discussions with the group and a comment on the first task-based PR. Only changes here are moving subpackages and modules into the new automl, fixing imports to work with this structure and fixing some dependencies in setup.py. * Fix doc building post automl subpackage refactor * Fix broken links in website post automl subpackage refactor * Fix broken links in website post automl subpackage refactor * Remove vw from test deps as this is breaking the build * Move default back to the top-level I'd moved this to automl as that's where it's used internally, but had missed that this is actually part of the public interface so makes sense to live where it was. * Re-add top level modules with deprecation warnings flaml.data, flaml.ml and flaml.model are re-added to the top level, being re-exported from flaml.automl for backwards compatability. Adding a deprecation warning so that we can have a planned removal later. * Fix model.py line-endings * WIP * WIP - Notes below Got to the point where the methods from AutoML are pulled to GenericTask. Started removing private markers and removing the passing of automl to these methods. Done with decide_split_type, started on prepare_data. Need to do the others after * Re-add generic_task * Fix tests: add Task.__str__ * Fix tests: test for ray.ObjectRef * Hotwire TS_Sklearn wrapper to fix test fail * Remove unused data size field from Task * Fix import for CLASSIFICATION in notebook * Update flaml/automl/data.py Co-authored-by: Chi Wang <wang.chi@microsoft.com> * Fix review comments * Fix task -> str in custom learner constructor * Remove unused CLASSIFICATION imports * Hotwire TS_Sklearn wrapper to fix test fail by setting optimizer_for_horizon == False * Revert changes to the automl_classification and pin FLAML version * Fix imports in reverted notebook * Fix FLAML version in automl notebooks * Fix ml.py line endings * Fix CLASSIFICATION task import in automl_classification notebook * Uncomment pip install in notebook and revert import Not convinced this will work because of installing an older version of the package into the environment in which we're running the tests, but let's see. * Revert c6a5dd1a0 * Revert "Revert c6a5dd1a0" This reverts commit e55e35adea03993de87b23f092b14c6af623d487. * Black format model.py * Bump version to 1.1.2 in automl_xgboost * Add docstrings to the Task ABC * Fix import in custom_learner * fix 'optimize_for_horizon' for ts_sklearn * remove debugging print statements * Check for is_forecast() before is_classification() in decide_split_type * Attempt to fix formatting fail * Another attempt to fix formatting fail * And another attempt to fix formatting fail * Add type annotations for task arg in signatures and docstrings * Fix formatting * Fix linting --------- Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu> Co-authored-by: EgorKraevTransferwise <egor.kraev@transferwise.com> Co-authored-by: Chi Wang <wang.chi@microsoft.com> Co-authored-by: Kevin Chen <chenkevin.8787@gmail.com> |