neighbors.models.NNMF_sgd

The non-negative matrix factorization algorithm tries to decompose a users x items matrix into two additional matrices: users x factors and factors x items.

Training is performed via stochastic-gradient-descent and continues until convergence or the maximum number of iterations has been reached. Unlike NNMF_mult errors during training are used to update latent factors separately for each user/item combination. Additionally this implementation is more flexible as it supports hyperparameters for various kinds of regularization at the cost of increased computation time.

The number of factors, convergence, and maximum iterations can be controlled with the n_factors, tol, and max_iterations arguments to the .fit method. By default the number of factors = the number items.

random_state does not control the sgd fit, only the initialization of the factor matrices

Important Note: model fitting can be highly sensitive to the regularization hyper-parameters passed to .fit. These hyper-parameters control the amount of regularization used when learning user and item factors and biases. By default no regularization is performed. For some combinations of hyper-parameters (e.g. large user_fact_reg and small item_fact_reg) latent vectors can blow up to infinity producing NaNs in model estimates. Model fitting will not fail in these cases so caution should be taken when making use of hyper-parameters.

Source code in neighbors/models.py

class NNMF_sgd(BaseNMF):
    """
    The non-negative matrix factorization algorithm tries to decompose a users x items matrix into two additional matrices: users x factors and factors x items.

    Training is performed via stochastic-gradient-descent and continues until convergence or the maximum number of iterations has been reached. Unlike `NNMF_mult` errors during training are used to update latent factors *separately* for each user/item combination. Additionally this implementation is more flexible as it supports hyperparameters for various kinds of regularization at the cost of increased computation time.

    The number of factors, convergence, and maximum iterations can be controlled with the `n_factors`, `tol`, and `max_iterations` arguments to the `.fit` method. By default the number of factors = the number items.

    `random_state` does not control the sgd fit, only the initialization of the factor matrices

    **Important Note**: model fitting can be highly sensitive to the regularization hyper-parameters passed to `.fit`. These hyper-parameters control the amount of regularization used when learning user and item factors and biases. By default *no regularization* is performed. For some combinations of hyper-parameters (e.g. large `user_fact_reg` and small `item_fact_reg`) latent vectors can blow up to infinity producing `NaNs` in model estimates. Model fitting will not fail in these cases so **caution** should be taken when making use of hyper-parameters.

    """

    def __init__(
        self, data, mask=None, n_mask_items=None, verbose=True, random_state=None
    ):
        """
        Args:
            data (pd.DataFrame): users x items dataframe
            mask (pd.DataFrame, optional): A boolean dataframe used to split the data into 'observed' and 'missing' datasets. Defaults to None.
            n_mask_items (int/float, optional): number of items to mask out, while the rest are treated as observed; Defaults to None.
            data_range (int/float, optional): max - min of the data; Default computed from the input data. This is useful to set manually in case the input data do not span the full range of possible values
            random_state (None, int, RandomState): a seed or random state used for all internal random operations (e.g. randomly mask half the data given n_mask_item = .05). Passing None will generate a new random seed. Default None.
            verbose (bool; optional): print any initialization warnings; Default True

        """
        super().__init__(
            data, mask, n_mask_items, random_state=random_state, verbose=verbose
        )
        self.n_factors = None

    def __repr__(self):
        return f"{super().__repr__()[:-1]}, n_factors={self.n_factors})"

    def fit(
        self,
        n_factors=None,
        item_fact_reg=0.0,
        user_fact_reg=0.0,
        item_bias_reg=0.0,
        user_bias_reg=0.0,
        learning_rate=0.001,
        n_iterations=1000,
        tol=1e-6,
        verbose=False,
        dilate_by_nsamples=None,
        clip_predictions=True,
        **kwargs,
    ):
        """
        Fit NNMF collaborative filtering model using stochastic-gradient-descent. **Note:** Some combinations of fit parameters may lead to degenerate fits due to use and item vectors converging to infinity. Because no constraints are imposed on the values these parameters can take, please adjust them with caution. If you encounter NaNs in your predictions it's likely because of the specific combination of parameters you chose and you can try refitting with the default settings (i.e. no regularization and learning rate = 0.001). Use `verbose=True` to help determine at what iteration these degenerate fits occur.

        Args:
            n_factors (int, optional): number of factors to learn. Defaults to None which includes all factors.
            item_fact_reg (float, optional): item factor regularization to apply. Defaults to 0.0.
            user_fact_reg (float, optional): user factor regularization to apply. Defaults to 0.0.
            item_bias_reg (float, optional): item factor bias term to apply. Defaults to 0.0.
            user_bias_reg (float, optional): user factor bias term to apply. Defaults to 0.0.
            learning_rate (float, optional): how quickly to integrate errors during training. Defaults to 0.001.
            n_iterations (int, optional): total number of training iterations if convergence is not achieved. Defaults to 5000.
            tol (float, optional): Convergence criteria. Model is considered converged if the change in error during training < tol. Defaults to 0.001.
            verbose (bool, optional): print information about training. Defaults to False.
            dilate_by_nsamples (int, optional): How many items to dilate by prior to training. Defaults to None.
            clip_predictions (bool, optional): clip predictions to the observed rating range, since the unconstrained bias terms can otherwise push predictions outside it (e.g. negative values despite all-positive ratings). This is the same approach the [Surprise](https://surpriselib.com/) package takes when making predictions. Defaults to True.
        """

        # Call parent fit which acts as a guard for non-masked data
        super().fit()

        # initialize variables
        n_users, n_items = self.data.shape

        if (
            isinstance(n_factors, int) and (n_factors > n_items and n_factors > n_users)
        ) or isinstance(n_factors, np.floating):
            raise TypeError("n_factors must be an integer < number of items and users")

        if n_factors is None:
            n_factors = min([n_users, n_items])

        self.n_factors = n_factors
        self.clip_predictions = clip_predictions
        self.item_fact_reg = item_fact_reg
        self.user_fact_reg = user_fact_reg
        self.item_bias_reg = item_bias_reg
        self.user_bias_reg = user_bias_reg
        self.error_history = []

        # Perform dilation if requested
        self.dilate_mask(n_samples=dilate_by_nsamples)

        # Get indices of training data to compute; np.nonzero returns a tuple of row and column indices that when iterated over simultaneosly yield the [row_index, col_index] of each training observation
        if self.is_mask_dilated:
            row_indices, col_indices = self.dilated_mask.values.nonzero()
        else:
            row_indices, col_indices = self.mask.values.nonzero()

        # Convert tuples cause numba complains
        row_indices, col_indices = np.array(row_indices), np.array(col_indices)

        # Initialize global, user, and item biases and latent vectors
        self.global_bias = self.masked_data.mean().mean()
        self.user_bias = np.zeros(n_users)
        self.item_bias = np.zeros(n_items)

        # Initialize random values oriented these as user x factor, factor x item
        self.user_vecs = np.abs(
            self.random_state.normal(scale=1.0 / n_factors, size=(n_users, n_factors))
        )
        self.item_vecs = np.abs(
            self.random_state.normal(scale=1.0 / n_factors, size=(n_factors, n_items))
        )

        X = self.masked_data.to_numpy()

        # Generate seed for shuffling within sgd
        seed = self.random_state.randint(np.iinfo(np.int32).max)

        # Run SGD
        # Silence numba warning until this issue gets fixed: https://github.com/numba/numba/issues/4585
        with warnings.catch_warnings():
            warnings.filterwarnings("ignore", category=NumbaPerformanceWarning)
            (
                error_history,
                converged,
                error_is_nan,
                n_iter,
                delta,
                norm_rmse,
                user_bias,
                user_vecs,
                item_bias,
                item_vecs,
            ) = sgd(
                X,
                seed,
                self.global_bias,
                self.data_range,
                tol,
                self.user_bias,
                self.user_vecs,
                self.user_bias_reg,
                self.user_fact_reg,
                self.item_bias,
                self.item_vecs,
                self.item_bias_reg,
                self.item_fact_reg,
                n_iterations,
                row_indices,
                col_indices,
                learning_rate,
                verbose,
            )
        # Save outputs to model
        (
            self.error_history,
            self.user_bias,
            self.user_vecs,
            self.item_bias,
            self.item_vecs,
        ) = (
            error_history,
            user_bias,
            user_vecs,
            item_bias,
            item_vecs,
        )

        self._n_iter = n_iter
        self._delta = delta
        self._norm_rmse = norm_rmse
        self.converged = converged
        self.error_is_nan = error_is_nan
        if verbose:
            if self.converged:
                print("\n\tCONVERGED!")
                print(f"\n\tFinal Iteration: {self._n_iter}")
                print(f"\tFinal Delta: {np.round(self._delta)}")
            elif self.error_is_nan:
                print("\tFAILED TO CONVERGE (predictions are NaN)")
                print(f"\n\tFinal Iteration: {self._n_iter}")
            else:
                print("\tFAILED TO CONVERGE (n_iter reached)")
                print(f"\n\tFinal Iteration: {self._n_iter}")
                print(f"\tFinal delta exceeds tol: {tol} <= {self._delta}")

            print(f"\tFinal Norm Error: {np.round(100 * norm_rmse, 2)}%")

        self._predict()
        self.is_fit = True

    def _predict(self):
        """Predict User's missing items using NNMF with stochastic gradient descent"""

        # user x factor * factor item + biases
        predictions = self.user_vecs @ self.item_vecs
        predictions = (
            (predictions.T + self.user_bias).T + self.item_bias + self.global_bias
        )
        if self.clip_predictions:
            predictions = np.clip(
                predictions,
                self.masked_data.min().min(),
                self.masked_data.max().max(),
            )
        self.predictions = pd.DataFrame(
            predictions, index=self.data.index, columns=self.data.columns
        )

__init__(self, data, mask=None, n_mask_items=None, verbose=True, random_state=None) special

Parameters:

Name	Type	Description	Default
data	pd.DataFrame	users x items dataframe	required
mask	pd.DataFrame	A boolean dataframe used to split the data into 'observed' and 'missing' datasets. Defaults to None.	None
n_mask_items	int/float	number of items to mask out, while the rest are treated as observed; Defaults to None.	None
data_range	int/float	max - min of the data; Default computed from the input data. This is useful to set manually in case the input data do not span the full range of possible values	required
random_state	None, int, RandomState	a seed or random state used for all internal random operations (e.g. randomly mask half the data given n_mask_item = .05). Passing None will generate a new random seed. Default None.	None
verbose	bool; optional	print any initialization warnings; Default True	True

Source code in neighbors/models.py

def __init__(
    self, data, mask=None, n_mask_items=None, verbose=True, random_state=None
):
    """
    Args:
        data (pd.DataFrame): users x items dataframe
        mask (pd.DataFrame, optional): A boolean dataframe used to split the data into 'observed' and 'missing' datasets. Defaults to None.
        n_mask_items (int/float, optional): number of items to mask out, while the rest are treated as observed; Defaults to None.
        data_range (int/float, optional): max - min of the data; Default computed from the input data. This is useful to set manually in case the input data do not span the full range of possible values
        random_state (None, int, RandomState): a seed or random state used for all internal random operations (e.g. randomly mask half the data given n_mask_item = .05). Passing None will generate a new random seed. Default None.
        verbose (bool; optional): print any initialization warnings; Default True

    """
    super().__init__(
        data, mask, n_mask_items, random_state=random_state, verbose=verbose
    )
    self.n_factors = None

fit(self, n_factors=None, item_fact_reg=0.0, user_fact_reg=0.0, item_bias_reg=0.0, user_bias_reg=0.0, learning_rate=0.001, n_iterations=1000, tol=1e-06, verbose=False, dilate_by_nsamples=None, clip_predictions=True, **kwargs)

Fit NNMF collaborative filtering model using stochastic-gradient-descent. Note: Some combinations of fit parameters may lead to degenerate fits due to use and item vectors converging to infinity. Because no constraints are imposed on the values these parameters can take, please adjust them with caution. If you encounter NaNs in your predictions it's likely because of the specific combination of parameters you chose and you can try refitting with the default settings (i.e. no regularization and learning rate = 0.001). Use verbose=True to help determine at what iteration these degenerate fits occur.

Parameters:

Name	Type	Description	Default
n_factors	int	number of factors to learn. Defaults to None which includes all factors.	None
item_fact_reg	float	item factor regularization to apply. Defaults to 0.0.	0.0
user_fact_reg	float	user factor regularization to apply. Defaults to 0.0.	0.0
item_bias_reg	float	item factor bias term to apply. Defaults to 0.0.	0.0
user_bias_reg	float	user factor bias term to apply. Defaults to 0.0.	0.0
learning_rate	float	how quickly to integrate errors during training. Defaults to 0.001.	0.001
n_iterations	int	total number of training iterations if convergence is not achieved. Defaults to 5000.	1000
tol	float	Convergence criteria. Model is considered converged if the change in error during training < tol. Defaults to 0.001.	1e-06
verbose	bool	print information about training. Defaults to False.	False
dilate_by_nsamples	int	How many items to dilate by prior to training. Defaults to None.	None
clip_predictions	bool	clip predictions to the observed rating range, since the unconstrained bias terms can otherwise push predictions outside it (e.g. negative values despite all-positive ratings). This is the same approach the Surprise package takes when making predictions. Defaults to True.	True

Source code in neighbors/models.py

def fit(
    self,
    n_factors=None,
    item_fact_reg=0.0,
    user_fact_reg=0.0,
    item_bias_reg=0.0,
    user_bias_reg=0.0,
    learning_rate=0.001,
    n_iterations=1000,
    tol=1e-6,
    verbose=False,
    dilate_by_nsamples=None,
    clip_predictions=True,
    **kwargs,
):
    """
    Fit NNMF collaborative filtering model using stochastic-gradient-descent. **Note:** Some combinations of fit parameters may lead to degenerate fits due to use and item vectors converging to infinity. Because no constraints are imposed on the values these parameters can take, please adjust them with caution. If you encounter NaNs in your predictions it's likely because of the specific combination of parameters you chose and you can try refitting with the default settings (i.e. no regularization and learning rate = 0.001). Use `verbose=True` to help determine at what iteration these degenerate fits occur.

    Args:
        n_factors (int, optional): number of factors to learn. Defaults to None which includes all factors.
        item_fact_reg (float, optional): item factor regularization to apply. Defaults to 0.0.
        user_fact_reg (float, optional): user factor regularization to apply. Defaults to 0.0.
        item_bias_reg (float, optional): item factor bias term to apply. Defaults to 0.0.
        user_bias_reg (float, optional): user factor bias term to apply. Defaults to 0.0.
        learning_rate (float, optional): how quickly to integrate errors during training. Defaults to 0.001.
        n_iterations (int, optional): total number of training iterations if convergence is not achieved. Defaults to 5000.
        tol (float, optional): Convergence criteria. Model is considered converged if the change in error during training < tol. Defaults to 0.001.
        verbose (bool, optional): print information about training. Defaults to False.
        dilate_by_nsamples (int, optional): How many items to dilate by prior to training. Defaults to None.
        clip_predictions (bool, optional): clip predictions to the observed rating range, since the unconstrained bias terms can otherwise push predictions outside it (e.g. negative values despite all-positive ratings). This is the same approach the [Surprise](https://surpriselib.com/) package takes when making predictions. Defaults to True.
    """

    # Call parent fit which acts as a guard for non-masked data
    super().fit()

    # initialize variables
    n_users, n_items = self.data.shape

    if (
        isinstance(n_factors, int) and (n_factors > n_items and n_factors > n_users)
    ) or isinstance(n_factors, np.floating):
        raise TypeError("n_factors must be an integer < number of items and users")

    if n_factors is None:
        n_factors = min([n_users, n_items])

    self.n_factors = n_factors
    self.clip_predictions = clip_predictions
    self.item_fact_reg = item_fact_reg
    self.user_fact_reg = user_fact_reg
    self.item_bias_reg = item_bias_reg
    self.user_bias_reg = user_bias_reg
    self.error_history = []

    # Perform dilation if requested
    self.dilate_mask(n_samples=dilate_by_nsamples)

    # Get indices of training data to compute; np.nonzero returns a tuple of row and column indices that when iterated over simultaneosly yield the [row_index, col_index] of each training observation
    if self.is_mask_dilated:
        row_indices, col_indices = self.dilated_mask.values.nonzero()
    else:
        row_indices, col_indices = self.mask.values.nonzero()

    # Convert tuples cause numba complains
    row_indices, col_indices = np.array(row_indices), np.array(col_indices)

    # Initialize global, user, and item biases and latent vectors
    self.global_bias = self.masked_data.mean().mean()
    self.user_bias = np.zeros(n_users)
    self.item_bias = np.zeros(n_items)

    # Initialize random values oriented these as user x factor, factor x item
    self.user_vecs = np.abs(
        self.random_state.normal(scale=1.0 / n_factors, size=(n_users, n_factors))
    )
    self.item_vecs = np.abs(
        self.random_state.normal(scale=1.0 / n_factors, size=(n_factors, n_items))
    )

    X = self.masked_data.to_numpy()

    # Generate seed for shuffling within sgd
    seed = self.random_state.randint(np.iinfo(np.int32).max)

    # Run SGD
    # Silence numba warning until this issue gets fixed: https://github.com/numba/numba/issues/4585
    with warnings.catch_warnings():
        warnings.filterwarnings("ignore", category=NumbaPerformanceWarning)
        (
            error_history,
            converged,
            error_is_nan,
            n_iter,
            delta,
            norm_rmse,
            user_bias,
            user_vecs,
            item_bias,
            item_vecs,
        ) = sgd(
            X,
            seed,
            self.global_bias,
            self.data_range,
            tol,
            self.user_bias,
            self.user_vecs,
            self.user_bias_reg,
            self.user_fact_reg,
            self.item_bias,
            self.item_vecs,
            self.item_bias_reg,
            self.item_fact_reg,
            n_iterations,
            row_indices,
            col_indices,
            learning_rate,
            verbose,
        )
    # Save outputs to model
    (
        self.error_history,
        self.user_bias,
        self.user_vecs,
        self.item_bias,
        self.item_vecs,
    ) = (
        error_history,
        user_bias,
        user_vecs,
        item_bias,
        item_vecs,
    )

    self._n_iter = n_iter
    self._delta = delta
    self._norm_rmse = norm_rmse
    self.converged = converged
    self.error_is_nan = error_is_nan
    if verbose:
        if self.converged:
            print("\n\tCONVERGED!")
            print(f"\n\tFinal Iteration: {self._n_iter}")
            print(f"\tFinal Delta: {np.round(self._delta)}")
        elif self.error_is_nan:
            print("\tFAILED TO CONVERGE (predictions are NaN)")
            print(f"\n\tFinal Iteration: {self._n_iter}")
        else:
            print("\tFAILED TO CONVERGE (n_iter reached)")
            print(f"\n\tFinal Iteration: {self._n_iter}")
            print(f"\tFinal delta exceeds tol: {tol} <= {self._delta}")

        print(f"\tFinal Norm Error: {np.round(100 * norm_rmse, 2)}%")

    self._predict()
    self.is_fit = True