# 计算机代写|机器学习代写machine learning代考|COMP3670

## 计算机代写|机器学习代写machine learning代考|Model Complexity and Regularization

So far we have talked vaguely about what it means for a model to be ‘too complex’ (or too simple) and suggested that we should choose a model that is complex enough to fit the data but simple enough not to overfit. This idea is often referred to as Occam’s Razor, a philosophical principle which states that among several alternate hypotheses that explain some phenomenon, one should favor the simplest.

However, for these notions to be useful, we must be precise about what it means for a model to be ‘complex.’ We would like to define complexity in terms of the parameters $\theta$, such that given a fixed set of features and labels, we could select the ‘simplest’ $\theta$ that adequately explains (or models) the data.
We will discuss two candidate notions of ‘simplicity’ as follows:
(i) A simple model is one that includes only a few terms, that is, in which only a few values $\theta_k$ are nonzero.
(ii) A simple model is one in which all terms are about equally important, that is, one in which particularly large values of $\theta_k$ are rare.

These two potential notions of ‘complexity’ are captured by the following expressions:
\begin{aligned} &\Omega_1(\theta)=|\theta|_1=\sum_k\left|\theta_k\right|, \ &\Omega_2(\theta)=|\theta|_2^2=\sum_k \theta_k^2, \end{aligned}
that is, the sum of absolute values and the sum of squares, also called the $\ell_1$ and (squared) $\ell_2$ norms of $\theta$. We state without proof that these expressions penalize models that have many nonzero parameters (eq. (3.29)) or large parameters (eq. (3.30)), though we further characterize their behavior later.

## 计算机代写|机器学习代写machine learning代考|Regularization

In order to fit a model which simultaneously explains the data but is not overly complex (corresponding to our goal above), we write down a new objective that combines our original accuracy objective with one of the complexity expressions above (in this case the squared $\ell_2$ norm). For a regression model we add the regularizer to the expression from Equation (2.16):
$$\underbrace{\frac{1}{|y|} \sum_{i=1}^{|y|}\left(x_i \cdot \theta-y_i\right)^2}{\text {accuracy }}+\underbrace{\sum_k \theta_k^2}{\text {model complexity } |\left.\theta\right|2 ^2} .$$ For a classification model, we subtract the regularizer, since we seek to maximize accuracy rather than minimizing error (so we maximize $-\lambda|\theta|_2^2$ rather than minimizing $\lambda|\theta|_2^2$ ): $$\sum_i-\log \left(1+e^{-x_i \cdot \theta}\right)+\sum{y_i=0}-x_i \cdot \theta-\lambda|\theta|_2^2 .$$
This procedure-where we add a penalty term to control model complexityis known as regularization; the parameter $\lambda$, which controls the extent to which complexity is penalized, is termed a regularization parameter.

Note that we can straightforwardly adapt the derivatives (from eqs. (2.54) and (3.9)) to include the regularization term, $\lambda|\theta|_2^2$ by noting that $\frac{\partial}{\partial \theta_k}$ $\lambda|\theta|_2^2=2 \lambda \theta_k$.

# 机器学习代考

## 计算机代写|机器学习代写machine learning代考|Model Complexity and Regularization

. Model Complexity and Regularization

(i)一个简单模型是一个只包含少数项的模型，也就是说，其中只有少数值$\theta_k$是非零。
(ii)一个简单模型是一个所有项都同等重要的模型，也就是说，其中$\theta_k$特别大的值是罕见的

\begin{aligned} &\Omega_1(\theta)=|\theta|_1=\sum_k\left|\theta_k\right|, \ &\Omega_2(\theta)=|\theta|_2^2=\sum_k \theta_k^2, \end{aligned}

## 计算机代写|机器学习代写machine learning代考|Regularization

.

$$\underbrace{\frac{1}{|y|} \sum_{i=1}^{|y|}\left(x_i \cdot \theta-y_i\right)^2}{\text {accuracy }}+\underbrace{\sum_k \theta_k^2}{\text {model complexity } |\left.\theta\right|2 ^2} .$$对于一个分类模型，我们减去正则化器，因为我们寻求最大化的精度而不是最小化的错误(所以我们最大化$-\lambda|\theta|_2^2$而不是最小化$\lambda|\theta|_2^2$): $$\sum_i-\log \left(1+e^{-x_i \cdot \theta}\right)+\sum{y_i=0}-x_i \cdot \theta-\lambda|\theta|_2^2 .$$

myassignments-help数学代考价格说明

1、客户需提供物理代考的网址，相关账户，以及课程名称，Textbook等相关资料~客服会根据作业数量和持续时间给您定价~使收费透明，让您清楚的知道您的钱花在什么地方。

2、数学代写一般每篇报价约为600—1000rmb，费用根据持续时间、周作业量、成绩要求有所浮动(持续时间越长约便宜、周作业量越多约贵、成绩要求越高越贵)，报价后价格觉得合适，可以先付一周的款，我们帮你试做，满意后再继续，遇到Fail全额退款。

3、myassignments-help公司所有MATH作业代写服务支持付半款，全款，周付款，周付款一方面方便大家查阅自己的分数，一方面也方便大家资金周转，注意:每周固定周一时先预付下周的定金，不付定金不予继续做。物理代写一次性付清打9.5折。

Math作业代写、数学代写常见问题

myassignments-help擅长领域包含但不是全部: