Programming/Tensorflow2017. 8. 2. 20:00

Learning rate

Overshooting : Learning rate의 값이 너무 커 튕겨나감

Takes too long : Learning rate의 값이 너무 작아 바닥까지 못내려감

 

Solution!

- Data(x) preprocessing

Original data -> Zero-centered-data -> Normalized data

- Standardiztion

X_std[i,0] = (X[i,0] - X[i,0].mean()) / X[i,0].std()

 

Overfitting

- Our model is very good with training data set

+ BUT not good at test dataset or in real use

 

Solution!

- More training data!

- Reduce the number of features

- Regularization

 

Regularization?

- Let's not have too big numbers in the weight

12reg = 0.001 * tf.reduce_sum(tf.square(w))

 

=================================================================

 

 

 

=================================================================

Posted by BadSchool