$How do I start using a trained neural network in R . When I try to put new data I get an error message that "y" is missing. Of course its missing, "y" is the dependent variable I want predicted.$

Question

$How do I start using a trained neural network in R . When I try to put new data I get an error message that "y" is missing. Of course its missing, "y" is the dependent variable I want predicted.$

Here is the code I used for training the model
library(neuralnet)

getwd()

d1 = read.csv("mydata.csv")

d1_dummy<- d1 %>% dplyr::rename( y=TOT )

d1_dummy

normalize <- function(x)

{maxval<<-max(x)

minval<<-min(x)

return ((x -minval) / (maxval -minval))}

denormalize <- function(x) {return( x*(maxval-minval) + minval )}

ddnorm <- lapply(d1_dummy,normalize) %>% as_tibble()

f <- as.formula(paste( "y", paste( names(ddnorm)[ -ncol(ddnorm) ], collapse = " + ") ,sep=" ~ " ) )

m_4_f<-neuralnet::neuralnet( f,data=ddnorm, hidden=c(5,4,3), rep=5, err.fct="sse", act.fct= 'logistic' )

class(m_4_f)

m_4_pred<-predict(m_4_f,newdata=ddnorm) %>% sapply(denormalize)

gg_input<-tibble(Predict=m_4_pred,Actual=d1_dummy$y)

with(gg_input, cor(Predict,Actual) )

plot(m_4_pred,col="blue")

points(d1_dummy$y,col="red")

forecast::accuracy(m_4_pred,x=d1_dummy$y)

Regression

Thesportspimp

26

Report

Mathe

0

Could you upload the dataset?
Thesportspimp

0

Yes give me a moment
Thesportspimp

0

I have attached the training data
Thesportspimp

0

Ive also attached the new data (LIVE2.csv)
Thesportspimp

0

Did the uploads work?
Thesportspimp

0

Rephrase Did you receive the uploads?
- Kav10
  
  0
  
  I do not see any files. But no need for the files yet. I will tell you what’s not working with your code and how you can fix it.
- Kav10
  
  0
  
  On the plane now, will submit the solution later tonight.
Thesportspimp

+1

Thank you very much

Answer

The answer is accepted.

Join Matchmaticians Affiliate Marketing Program to earn up to a 50% commission on every question that your affiliated users ask or answer.

Thesportspimp

0

Ive also attached the new data (LIVE2.csv)
Thesportspimp

0

Rephrase Did you receive the uploads?

Kav10

0

I do not see any files. But no need for the files yet. I will tell you what’s not working with your code and how you can fix it.

Kav10

0

On the plane now, will submit the solution later tonight.
Kav10

0

I do not see any files. But no need for the files yet. I will tell you what’s not working with your code and how you can fix it.
Kav10

0

On the plane now, will submit the solution later tonight.

Answer 1

Answers can only be viewed under the following conditions:

The questioner was satisfied with and accepted the answer, or
The answer was evaluated as being 100% correct by the judge.

View the answer

Hey Thesportspimp,

The error you are getting is because predict() for neuralnet in R does not use "newdata=" like in many other models (e.g. lm). Instead, you need to manually prepare a matrix of inputs and use the compute() function instead of predict(). Compute() is the correct function for predicting with neuralnet models.

I mean you can still use the predict() function, but need to ensure that the new data is preprocessed in the same way as the training data. When predicting, only input variables should be provided. The predict() function expects the same structure of input data as the training data, including the dependent variable "y", even though it won't be used for prediction.

Predict() is a generic function in R that works with many types of models and for neuralnet, it internally calls compute().

On the other hand, compute() is specific to the neuralnet package and directly computes the output of the neural network for the given input data.

So, you should first ensure that the new data does not contain the dependent variable "y", because you are predicting it. Since "y" is what you want to predict, it should not be included in the new data.

Then, use compute() instead of predict() for making predictions.

You should also normalize data before. The model expects input data in the same normalized format as training data.

These should resolve the error you are getting and give you correct predictions. Let me know if you have any questions!

Try this please with your data:

newdata <- read.csv("LIVE2.csv")
d1_dummy <- newdata %>% dplyr::rename(y = TOT)

normalize <- function(x) {
maxval <<- max(x)
minval <<- min(x)
return ((x - minval) / (maxval - minval))
}

denormalize <- function(x) {
return(x * (maxval - minval) + minval)
}

ddnorm <- lapply(d1_dummy, normalize) %>% as_tibble()
f <- as.formula(paste("y", paste(names(ddnorm)[-ncol(ddnorm)], collapse = " + "), sep = " ~ "))

new_data <- ddnorm %>% select(-y)
m_4_comp <- compute(m_4_f, new_data)$net.result %>% sapply(denormalize)

gg_input <- tibble(Compute = m_4_comp)

Kav10

2.1K

Thesportspimp

0

I will try it now and let you know if that works, Thank You
- Kav10
  
  0
  
  Sounds good.
Thesportspimp

0

So after ddnorm <- lapply(d1_dummy,normalize) %>% as_tibble() Code should read f <- as.formula(paste( "y", paste( names(ddnorm)[ -ncol(ddnorm) ], collapse = " + ") ,sep=" ~ " ) ) m_4_f<-neuralnet::neuralnet( f,data=ddnorm, hidden=c(5,4,3), rep=5, err.fct="sse", act.fct= 'logistic' ) m_4_comp<-compute(m_4_f,) %>% sapply(denormalize) gg_input<-tibble(Compute=m_4_comp)
- Kav10
  
  0
  
  The compute() function requires input data (excluding y). Add these (add the first line, replace the one you have with the second line): new_data <- ddnorm %>% select(-y) m_4_comp <- compute(m_4_f, new_data)$net.result %>% sapply(denormalize)
Thesportspimp

0

> newdata = read.csv("LIVE2.csv") > d1_dummy<- newdata %>% dplyr::rename( y=TOT ) > normalize <- function(x) + {maxval<<-max(x) + minval<<-min(x) + return ((x -minval) / (maxval -minval)) + } > denormalize <- function(x) {return( x*(maxval-minval) + minval )} > ddnorm <- lapply(d1_dummy,normalize) %>% as_tibble() > f <- as.formula(paste( "y", paste( names(ddnorm)[ -ncol(ddnorm) ], collapse = " + ") ,sep=" ~ " ) ) >> new_data <- ddnorm %>% select(-y)
Thesportspimp

0

> m_4_comp <- compute(m_4_f, new_data)$net.result %>% sapply(denormalize) > gg_input<-tibble(Compute=m_4_comp) > print(compute) function (x, covariate, rep = 1) { pred <- predict.nn(x, newdata = covariate, rep = rep, all.units = TRUE) for (i in 1:(length(pred) - 1)) { pred[[i]] <- cbind(1, pred[[i]]) } list(neurons = pred[-length(pred)], net.result = pred[[length(pred)]]) } Is the result I got
- Kav10
  
  0
  
  Did it work?
Thesportspimp

0

Sorry it wont let me seperate the lines of code but, I still did it wrong and got the error message at the end............function (x, covariate, rep = 1) { pred <- predict.nn(x, newdata = covariate, rep = rep, all.units = TRUE) for (i in 1:(length(pred) - 1)) { pred[[i]] <- cbind(1, pred[[i]]) } list(neurons = pred[-length(pred)], net.result = pred[[length(pred)]]) }
Thesportspimp

0

Im new to this and its showing, could you write the code in order, thats giving me the most trouble. I will tip you extra for your time
- Kav10
  
  0
  
  Yes, it is really hard to follow these in a comment. I will write the code in order and add to the solution above (not in the comments).
- Kav10
  
  0
  
  Added. Please check.
Thesportspimp

0

Doing it right now
Thesportspimp

0

That works, THANK YOU SO MUCH
Thesportspimp

0

I put in a 20 dollar tip, thanks again
- Kav10
  
  0
  
  I don’t think the tip went through. Please check your account.
Kav10

0

Great! Thank you.
Kav10

0

For next times, you can ask a direct question. Click on my profile and then click on ask direct question.
Thesportspimp

0

You should have the tip now I will ask you direct from now for sure You were very helpful
- Kav10
  
  0
  
  Sounds good. For your next questions, to determine appropriate bounty, just remember that the website gets 20% of the bounty and tip as commission.

$How do I start using a trained neural network in R . When I try to put new data I get an error message that "y" is missing. Of course its missing, "y" is the dependent variable I want predicted.$

Answer

Related Questions

Search