The following error was resolved:
: Input 0 is incompatible with layer conv1d_1: expected ndim=3, found ndim=4
: Error when checking target: expected dense_3 to have 3 dimensions, but got array with …
: Input 0 is incompatible with layer conv1d_1: expected ndim=3, found ndim=4
Error Code:
(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape))
or
(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1:])))
This is because the dimensions of the model inputs are wrong, in using tensorflow based keras the input_shape of cov1d is two dimensional and should be:
1. shape of reshape x_train
x_train=x_train.reshape((x_train.shape[0],x_train.shape[1],1))
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1],1))
2、Change input_shape
model = Sequential()
(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1],1)))
The Great God Original Article:
The input shape is wrong, it should be input_shape = (1, 3253) for Theano or (3253, 1) for TensorFlow. The input shape doesn't include the number of samples.
Then you need to reshape your data to include the channels axis:
x_train = x_train.reshape((500000, 1, 3253))
Or move the channels dimension to the end if you use TensorFlow. After these changes it should work.
: Error when checking target: expected dense_3 to have 3 dimensions, but got array with …
This problem occurs because the dimension of ylabel doesn't match x_train x_test, and since you reshape both x_train x_test, you need to reshape y as well.
Solution:
Also change the shape of the ylabel against x_train
t_train=t_train.reshape((t_train.shape[0],1))
t_test = t_test.reshape((t_test.shape[0],1))
Attachment:
Modified code:
import warnings
("ignore")
import os
["CUDA_VISIBLE_DEVICES"] = "0"
import pandas as pd
import numpy as np
import matplotlib
# ('Agg')
import as plt
from sklearn.model_selection import train_test_split
from sklearn import preprocessing
from import Sequential
from import Dense, Dropout, BatchNormalization, Activation, Flatten, Conv1D
from import LearningRateScheduler, EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
from keras import optimizers
from import l2
from import load_model
df_train = pd.read_csv('./input/train_V2.csv')
df_test = pd.read_csv('./input/test_V2.csv')
df_train.drop(df_train.index[[2744604]],inplace=True)# Remove the nan value
df_train["distance"] = df_train["rideDistance"]+df_train["walkDistance"]+df_train["swimDistance"]
# df_train["healthpack"] = df_train["boosts"] + df_train["heals"]
df_train["skill"] = df_train["headshotKills"]+df_train["roadKills"]
df_test["distance"] = df_test["rideDistance"]+df_test["walkDistance"]+df_test["swimDistance"]
# df_test["healthpack"] = df_test["boosts"] + df_test["heals"]
df_test["skill"] = df_test["headshotKills"]+df_test["roadKills"]
df_train_size = df_train.groupby(['matchId','groupId']).size().reset_index(name='group_size')
df_test_size = df_test.groupby(['matchId','groupId']).size().reset_index(name='group_size')
df_train_mean = df_train.groupby(['matchId','groupId']).mean().reset_index()
df_test_mean = df_test.groupby(['matchId','groupId']).mean().reset_index()
df_train = (df_train, df_train_mean, suffixes=["", "_mean"], how='left', on=['matchId', 'groupId'])
df_test = (df_test, df_test_mean, suffixes=["", "_mean"], how='left', on=['matchId', 'groupId'])
del df_train_mean
del df_test_mean
df_train = (df_train, df_train_size, how='left', on=['matchId', 'groupId'])
df_test = (df_test, df_test_size, how='left', on=['matchId', 'groupId'])
del df_train_size
del df_test_size
target = 'winPlacePerc'
train_columns = list(df_test.columns)
""" remove some columns """
train_columns.remove("Id")
train_columns.remove("matchId")
train_columns.remove("groupId")
train_columns_new = []
for name in train_columns:
if '_' in name:
train_columns_new.append(name)
train_columns = train_columns_new
# print(train_columns)
X = df_train[train_columns]
Y = df_test[train_columns]
T = df_train[target]
del df_train
x_train, x_test, t_train, t_test = train_test_split(X, T, test_size = 0.2, random_state = 1234)
# scaler = (feature_range=(-1, 1)).fit(x_train)
scaler = ().fit(x_train)
x_train = (x_train)
x_test = (x_test)
Y = (Y)
x_train=x_train.reshape((x_train.shape[0],x_train.shape[1],1))
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1],1))
t_train=t_train.reshape((t_train.shape[0],1))
t_test = t_test.reshape((t_test.shape[0],1))
model = Sequential()
(Conv1D(8, kernel_size=3, strides=1, padding='same', input_shape=(x_train.shape[1],1)))
(BatchNormalization())
(Conv1D(8, kernel_size=3, strides=1, padding='same'))
(Conv1D(16, kernel_size=3, strides=1, padding='valid'))
(BatchNormalization())
(Conv1D(16, kernel_size=3, strides=1, padding='same'))
(Conv1D(32, kernel_size=3, strides=1, padding='valid'))
(BatchNormalization())
(Conv1D(32, kernel_size=3, strides=1, padding='same'))
(Conv1D(32, kernel_size=3, strides=1, padding='same'))
(Conv1D(64, kernel_size=3, strides=1, padding='same'))
(Activation('tanh'))
(Flatten())
(Dropout(0.5))
# (Dropout(0.25))
(Dense(512,kernel_initializer='he_normal', activation='relu', W_regularizer=l2(0.01)))
(Dense(128,kernel_initializer='he_normal', activation='relu', W_regularizer=l2(0.01)))
(Dense(1, kernel_initializer='normal', activation='sigmoid'))
(lr=0.01, epsilon=1e-8, decay=1e-4)
(optimizer=optimizer, loss='mse', metrics=['mae'])
()
ng = EarlyStopping(monitor='val_mean_absolute_error', mode='min', patience=4, verbose=1)
# model_checkpoint = ModelCheckpoint(filepath='best_model.h5', monitor='val_mean_absolute_error', mode = 'min', save_best_only=True, verbose=1)
# reduce_lr = ReduceLROnPlateau(monitor='val_mean_absolute_error', mode = 'min',factor=0.5, patience=3, min_lr=0.0001, verbose=1)
history = (x_train, t_train,
validation_data=(x_test, t_test),
epochs=30,
batch_size=32768,
callbacks=[early_stopping],
verbose=1)predict(Y)
pred = ()
Additional knowledge:Keras Conv1d Parameters and Input/Output Details
Conv1d(in_channels，out_channels，kernel_size，stride=1，padding=0，dilation=1，groups=1，bias=True)
filters:number of convolution kernels (i.e., dimension of the output)
kernel_size: integer or list/tuple consisting of a single integer, length of the null or time domain window of the convolution kernel
strides: integers or lists/tuples of individual integers, for the step size of the convolution. Any strides that are not 1 are incompatible with any dilation_rata that are not 1
padding: a make-up-0 strategy, as "valid", "same" or "casual", "casual " will produce causal (inflated) convolution, i.e. output[t] does not depend on input[t+1:]. Useful when modeling temporal signals that cannot violate the order of events. "valid" means only valid convolution, i.e. no processing of boundary data. "same" means that the convolution result at the boundary is preserved, which usually results in the output shape being the same as the input shape.
activation: activation function, a predefined activation function name, or an element-by-element Theano function. If this function is not specified, no activation function will be used (i.e., the linear activation function is used: a(x)=x).
(Conv1D(filters=nn_params["input_filters"],
kernel_size=nn_params["filter_length"],
strides=1,
padding='valid',
activation=nn_params["activation"],
kernel_regularizer=l2(nn_params["reg"])))
Example: input dimension is (None, 1000, 4)
The first dimension:None
Second dimension:
output_length = int((input_length - nn_params["filter_length"] + 1))
In this case for:
output_length = （1000 + 2*padding - filters +1）/ strides = （1000 + 2*0 -32 +1）/1 = 969
The third dimension:filters
The above article to solve the input problem of keras using cov1D function is all I have shared with you, I hope it can give you a reference, and I hope you will support me more.