:orphan:

Tutorial
================

The Tutorial section aims to provide working code samples demonstrating 
how to use the ``abess`` library to solve real world issues. 
In the following pages, the ``abess`` Python package is used for illustration. 
The counterpart for R package is available at `here`_. 

.. _here: https://abess-team.github.io/abess/articles/



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. thumbnail-parent-div-close

.. raw:: html

    </div>

Generalized Linear Model
-------------------------



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, we are going to demonstrate how to use the abess package to carry out best su...">

.. only:: html

  .. image:: /auto_gallery/1-glm/images/thumb/sphx_glr_plot_1_LinearRegression_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_1-glm_plot_1_LinearRegression.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Linear Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Classification: Logistic Regression and Beyond">

.. only:: html

  .. image:: /auto_gallery/1-glm/images/thumb/sphx_glr_plot_2_LogisticRegression_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_1-glm_plot_2_LogisticRegression.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Classification: Logistic Regression and Beyond</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Multi-Response Linear Regression">

.. only:: html

  .. image:: /auto_gallery/1-glm/images/thumb/sphx_glr_plot_3_MultiTaskLearning_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_1-glm_plot_3_MultiTaskLearning.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Multi-Response Linear Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Survival Analysis: Cox Regression">

.. only:: html

  .. image:: /auto_gallery/1-glm/images/thumb/sphx_glr_plot_4_CoxRegression_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_1-glm_plot_4_CoxRegression.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Survival Analysis: Cox Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Positive Responses: Poisson &amp; Gamma Regressions">

.. only:: html

  .. image:: /auto_gallery/1-glm/images/thumb/sphx_glr_plot_5_PossionGammaRegression_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_1-glm_plot_5_PossionGammaRegression.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Positive Responses: Poisson & Gamma Regressions</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Power of abess Library: Empirical Comparison">

.. only:: html

  .. image:: /auto_gallery/1-glm/images/thumb/sphx_glr_plot_a1_power_of_abess_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_1-glm_plot_a1_power_of_abess.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Power of abess Library: Empirical Comparison</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="ABESS Algorithm: Details">

.. only:: html

  .. image:: /auto_gallery/1-glm/images/thumb/sphx_glr_plot_a2_abess_algorithm_details_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_1-glm_plot_a2_abess_algorithm_details.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">ABESS Algorithm: Details</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>

Principal Component Analysis
-------------------------------


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Principal Component Analysis">

.. only:: html

  .. image:: /auto_gallery/2-pca/images/thumb/sphx_glr_plot_6_PCA_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_2-pca_plot_6_PCA.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Principal Component Analysis</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Robust Principal Component Analysis">

.. only:: html

  .. image:: /auto_gallery/2-pca/images/thumb/sphx_glr_plot_7_RPCA_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_2-pca_plot_7_RPCA.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Robust Principal Component Analysis</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>

Advanced Generic Features
--------------------------

When analyzing the real world datasets, we may have the following targets:
 
1. identifying predictors when group structure are provided (a.k.a., **best group subset selection**);
2. certain variables must be selected when some prior information is given (a.k.a., **nuisance regression**);
3. selecting the weak signal variables when the prediction performance is mainly interested (a.k.a., **regularized best-subset selection**). 

These targets are frequently encountered in real world data analysis. 
Actually, in our methods, the targets can be properly handled by simply change some default arguments in the functions. 
In the following content, we will illustrate the statistic methods to reach these targets in a one-by-one manner, 
and give quick examples to show how to perform the statistic methods in ``LinearRegression`` and 
the same steps can be implemented in all methods. 

Besides, ``abess`` library is very flexible, i.e., users can flexibly control many internal computational components. 
Specifically, users can specify: (i) the division of samples in cross validation (a.k.a., **cross validation division**), 
(ii) specify the initial active set before splicing (a.k.a., **initial active set**), and so on.
We will also describe these in the following. 



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Best Subset of Group Selection">

.. only:: html

  .. image:: /auto_gallery/3-advanced-features/images/thumb/sphx_glr_plot_best_group_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_3-advanced-features_plot_best_group.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Best Subset of Group Selection</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Nuisance Regression">

.. only:: html

  .. image:: /auto_gallery/3-advanced-features/images/thumb/sphx_glr_plot_best_nuisance_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_3-advanced-features_plot_best_nuisance.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Nuisance Regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Regularized Best Subset Selection">

.. only:: html

  .. image:: /auto_gallery/3-advanced-features/images/thumb/sphx_glr_plot_best_regularized_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_3-advanced-features_plot_best_regularized.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Regularized Best Subset Selection</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Cross-Validation Division">

.. only:: html

  .. image:: /auto_gallery/3-advanced-features/images/thumb/sphx_glr_plot_cross_validation_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_3-advanced-features_plot_cross_validation.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Cross-Validation Division</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Initial Active Set">

.. only:: html

  .. image:: /auto_gallery/3-advanced-features/images/thumb/sphx_glr_plot_initial_active_set_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_3-advanced-features_plot_initial_active_set.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Initial Active Set</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>

Computational Tips
--------------------------
The generic splicing technique certifiably guarantees the best subset can be selected in a polynomial time. 
In practice, the computational efficiency can be improved to handle large scale datasets. 
The tips for computational improvement are applicable for: 

1. **ultra-high dimensional data** via

   - feature screening;
   - focus on important variables;
2. **large-sample data** via

   - golden-section searching;
   - early-stop scheme;
3. **sparse inputs** via

   - sparse matrix computation;
4. **specific models** via

   - covariance update for ``LinearRegression`` and ``MultiTaskRegression``;
   - quasi Newton iteration for ``LogisticRegression``, ``PoissonRegression``, ``CoxRegression``, etc.

More importantly, the technique in these tips can be use simultaneously. 
For example, ``abess`` allow algorithms to use both feature screening and golden-section searching such that 
algorithms can handle datasets with large-sample and ultra-high dimension. 
The following contents illustrate the above tips.

Besides, ``abess`` efficiently implements warm-start initialization and parallel computing, 
which are very useful for fast computing. 
To help use leverage them, we will also describe their implementation details in the following. 


.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Ultra-High dimensional data">

.. only:: html

  .. image:: /auto_gallery/4-computation-tips/images/thumb/sphx_glr_plot_large_dimension_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_4-computation-tips_plot_large_dimension.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Ultra-High dimensional data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Large-Sample Data">

.. only:: html

  .. image:: /auto_gallery/4-computation-tips/images/thumb/sphx_glr_plot_large_sample_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_4-computation-tips_plot_large_sample.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Large-Sample Data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Sparse Inputs">

.. only:: html

  .. image:: /auto_gallery/4-computation-tips/images/thumb/sphx_glr_plot_sparse_inputs_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_4-computation-tips_plot_sparse_inputs.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Sparse Inputs</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Specific Models">

.. only:: html

  .. image:: /auto_gallery/4-computation-tips/images/thumb/sphx_glr_plot_specific_models_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_4-computation-tips_plot_specific_models.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Specific Models</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>

Connect to Popular Libraries with ``scikit-learn`` API
---------------------------------------------------------

This part is intended to present all possible connection between ``abess`` and 
other popular Python libraries via the ``scikit-learn`` interface. 
It is keep developing and more examples will come up soon. 
Contributions for this part is extremely welcome! 



.. raw:: html

    <div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Work with scikit-learn">

.. only:: html

  .. image:: /auto_gallery/5-scikit-learn-connection/images/thumb/sphx_glr_plot_1_scikit_learn_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_5-scikit-learn-connection_plot_1_scikit_learn.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Work with scikit-learn</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Work with geomstats">

.. only:: html

  .. image:: /auto_gallery/5-scikit-learn-connection/images/thumb/sphx_glr_plot_2_geomstats_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_5-scikit-learn-connection_plot_2_geomstats.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Work with geomstats</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Work with DoubleML">

.. only:: html

  .. image:: /auto_gallery/5-scikit-learn-connection/images/thumb/sphx_glr_plot_3_double_machine_learning_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_5-scikit-learn-connection_plot_3_double_machine_learning.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Work with DoubleML</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Work with pyts">

.. only:: html

  .. image:: /auto_gallery/5-scikit-learn-connection/images/thumb/sphx_glr_plot_4_pyts_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_5-scikit-learn-connection_plot_4_pyts.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Work with pyts</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Work with DoWhy">

.. only:: html

  .. image:: /auto_gallery/5-scikit-learn-connection/images/thumb/sphx_glr_plot_5_dowhy_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_5-scikit-learn-connection_plot_5_dowhy.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Work with DoWhy</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Work with imbalanced-learn">

.. only:: html

  .. image:: /auto_gallery/5-scikit-learn-connection/images/thumb/sphx_glr_plot_6_imbalanced_learn_thumb.png
    :alt:

  :ref:`sphx_glr_auto_gallery_5-scikit-learn-connection_plot_6_imbalanced_learn.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Work with imbalanced-learn</div>
    </div>


.. thumbnail-parent-div-close

.. raw:: html

    </div>


.. toctree::
   :hidden:
   :includehidden:


   /auto_gallery/1-glm/index.rst
   /auto_gallery/2-pca/index.rst
   /auto_gallery/3-advanced-features/index.rst
   /auto_gallery/4-computation-tips/index.rst
   /auto_gallery/5-scikit-learn-connection/index.rst


.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-gallery

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download all examples in Python source code: auto_gallery_python.zip </auto_gallery/auto_gallery_python.zip>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download all examples in Jupyter notebooks: auto_gallery_jupyter.zip </auto_gallery/auto_gallery_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_