多进程执行 MySQL 报错 Packets out of order [ 2.0 版本 ]
代码如下
public function actionSend()
{
$userCount = Yii::$app->db_uums->createCommand('SELECT count(*) as count FROM `User` WHERE UserType in (1,12,26,27) AND Telephone NOT LIKE "%-%" AND length(UserName) = char_length(UserName);')->queryOne();
$count = $userCount['count'];
if (!file_exists($this->csvPath)) {
CommonFun::recursionMkDir($this->csvPath);
}
$totalNum = ceil($count/$this->step);
$childs = array();
for($i=0; $i< $totalNum; $i++) {
$pid = pcntl_fork();
if ($pid == -1) {
die('Could not fork');
} elseif($pid) {
echo "I'm the Parent $i\n";
$childs[] = $pid;
} else {
//子进程处理业务
$this->handelChildProcess($i,$count);
//$userList = Yii::$app->db_uums->createCommand('SELECT UserID,UserName,TrueName,`PassWord`, Gender,UserType,ShoolID,Email,Telephone,RegDate,inviteCode FROM `User` WHERE UserType in (1,12,26,27) AND Telephone NOT LIKE "%-%" AND length(UserName) = char_length(UserName) LIMIT 100 OFFSET 3000000')->queryAll();
}
}
while (count($childs) > 0) {
foreach ($childs as $key => $pid) {
$res = pcntl_waitpid($pid, $status, WNOHANG);
//-1代表error, 大于0代表子进程已退出,返回的是子进程的pid,非阻塞时0代表没取到退出子进程
if ($res == -1 || $res > 0) {
echo "$key=> $pid\n";
unset($childs[$key]);
}
}
sleep(1);
}
}
public function getChildUsers($start) {
$size = $this->size;
$userList = Yii::$app->db_uums->createCommand('SELECT UserID,UserName,TrueName,`PassWord`, Gender,UserType,ShoolID,Email,Telephone,RegDate,inviteCode FROM `User` WHERE UserType in (1,12,26,27) AND Telephone NOT LIKE "%-%" AND length(UserName) = char_length(UserName) LIMIT ' . $size . ' OFFSET ' . $start)->queryAll();
foreach ($userList as $value) {
yield $value;
}
}
public function handelChildProcess($childKey, $totalCount)
{
echo "process $childKey start \n";
$taskStartTime = microtime(true);
$pageTotal = ceil($this->step/$this->size);
for ($i=1; $i <= $pageTotal; $i++) {
//计算起始位置
$start = $childKey * $this->step + ($i-1) * $this->size;
if ($start > $totalCount) {
$lastTime = CommonFun::getElapsedTime($taskStartTime);
Yii::info("lastTime|process" . $childKey . $lastTime, __METHOD__);
echo "process $childKey end\n";
exit('超过总数了');
}
$userList = $this->getChildUsers($start);
foreach ($userList as $key => $value) {
$loginName = $value['UserName'];
$mobilePhone = $value['Telephone'];
$userId = $value['UserID'];
//$model = UserUums::findOne($userId);
//mod已经存在的用户 modExist.csv
if ($this->accountExist($loginName)) {
//$model->checkStatus = 2;
$this->writeRow($value,$this->csvPath . "/modExist" . $childKey);
}
//非法的手机号
elseif($mobilePhone && !CommonFun::is_mobile($mobilePhone)) {
//$model->checkStatus = 3;
$this->writeRow($value,$this->csvPath . '/modbadTel' . $childKey);
}
//mod中AccountName 不存在 但是mobilePhone已经存在 modTelExist.csv
elseif ($mobilePhone && CommonFun::is_mobile($mobilePhone) && $this->accountExist(($mobilePhone))) {
$this->writeRow($value,$this->csvPath . '/modTelExist' . $childKey);
//$model->checkStatus = 4;
}
else {
$this->writeRow($value, $this->csvPath . '/user' . $childKey);
//$model->checkStatus = 1;
}
//$this->saveUumsUser($model);
}
}
$endTime = microtime(true);
$lastTime = number_format($endTime- $taskStartTime, 4);
Yii::info("lastTime|process" . $childKey . $lastTime, __METHOD__);
echo "process $childKey end\n";
exit($lastTime);
}
执行之后报错,报错信息如下
process 0 start
I'm the Parent 0
process 1 start
I'm the Parent 1
Exception 'yii\db\Exception' with message 'PDOStatement::execute(): Premature end of data (mysqlnd_wireprotocol.c:1279)
The SQL being executed was: SELECT UserID,UserName,TrueName,`PassWord`, Gender,UserType,ShoolID,Email,Telephone,RegDate,inviteCode FROM `User` WHERE UserType in (1,12,26,27) AND Telephone NOT LIKE "%-%" AND length(UserName) = char_length(UserName) LIMIT 100 OFFSET 3000000'
in /data/www/bbbb/vendor/yiisoft/yii2/db/Schema.php:636
Stack trace:
#0 /data/www/bbbb/vendor/yiisoft/yii2/db/Command.php(917): yii\db\Schema->convertException(Object(yii\base\ErrorException), 'SELECT UserID,...')
#1 /data/www/bbbb/vendor/yiisoft/yii2/db/Command.php(362): yii\db\Command->queryInternal('fetchAll', NULL)
#2 /data/www/bbbb/console/controllers/UumsController.php(419): yii\db\Command->queryAll()
#3 /data/www/bbbb/console/controllers/UumsController.php(442): console\controllers\UumsController->getChildUsers(3000000)
#4 /data/www/bbbb/console/controllers/UumsController.php(397): console\controllers\UumsController->handelChildProcess(1, '3474770')
#5 [internal function]: console\controllers\UumsController->actionSend()
#6 /data/www/bbbb/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#7 /data/www/bbbb/vendor/yiisoft/yii2/base/Controller.php(156): yii\base\InlineAction->runWithParams(Array)
#8 /data/www/bbbb/vendor/yiisoft/yii2/console/Controller.php(128): yii\base\Controller->runAction('send', Array)
#9 /data/www/bbbb/vendor/yiisoft/yii2/base/Module.php(523): yii\console\Controller->runAction('send', Array)
#10 /data/www/bbbb/vendor/yiisoft/yii2/console/Application.php(180): yii\base\Module->runAction('uums/send', Array)
#11 /data/www/bbbb/vendor/yiisoft/yii2/console/Application.php(147): yii\console\Application->runAction('uums/send', Array)
#12 /data/www/bbbb/vendor/yiisoft/yii2/base/Application.php(380): yii\console\Application->handleRequest(Object(yii\console\Request))
#13 /data/www/bbbb/yii(27): yii\base\Application->run()
#14 {main}
Exception 'yii\db\Exception' with message 'Packets out of order. Expected 1 received 115. Packet size=7173493
The SQL being executed was: SELECT UserID,UserName,TrueName,`PassWord`, Gender,UserType,ShoolID,Email,Telephone,RegDate,inviteCode FROM `User` WHERE UserType in (1,12,26,27) AND Telephone NOT LIKE "%-%" AND length(UserName) = char_length(UserName) LIMIT 100 OFFSET 0'
in /data/www/bbbb/vendor/yiisoft/yii2/db/Schema.php:636
Stack trace:
#0 /data/www/bbbb/vendor/yiisoft/yii2/db/Command.php(917): yii\db\Schema->convertException(Object(yii\base\ErrorException), 'SELECT UserID,...')
#1 /data/www/bbbb/vendor/yiisoft/yii2/db/Command.php(362): yii\db\Command->queryInternal('fetchAll', NULL)
#2 /data/www/bbbb/console/controllers/UumsController.php(419): yii\db\Command->queryAll()
#3 /data/www/bbbb/console/controllers/UumsController.php(442): console\controllers\UumsController->getChildUsers(0)
#4 /data/www/bbbb/console/controllers/UumsController.php(397): console\controllers\UumsController->handelChildProcess(0, '3474770')
#5 [internal function]: console\controllers\UumsController->actionSend()
#6 /data/www/bbbb/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array(Array, Array)
#7 /data/www/bbbb/vendor/yiisoft/yii2/base/Controller.php(156): yii\base\InlineAction->runWithParams(Array)
#8 /data/www/bbbb/vendor/yiisoft/yii2/console/Controller.php(128): yii\base\Controller->runAction('send', Array)
#9 /data/www/bbbb/vendor/yiisoft/yii2/base/Module.php(523): yii\console\Controller->runAction('send', Array)
#10 /data/www/bbbb/vendor/yiisoft/yii2/console/Application.php(180): yii\base\Module->runAction('uums/send', Array)
#11 /data/www/bbbb/vendor/yiisoft/yii2/console/Application.php(147): yii\console\Application->runAction('uums/send', Array)
#12 /data/www/bbbb/vendor/yiisoft/yii2/base/Application.php(380): yii\console\Application->handleRequest(Object(yii\console\Request))
#13 /data/www/bbbb/yii(27): yii\base\Application->run()
#14 {main}
0=> 57658
1=> 57659
bzzear 补充于 2018-11-16 15:10
猜测可能的原因是子进程会继承主进程的数据库连接,当mysql返回数据时,这些子进程都可以通过这个连接读到数据
共 0 个回答
没有找到数据。
bzzear
注册时间:2017-04-20
最后登录:2021-04-18
在线时长:42小时17分
最后登录:2021-04-18
在线时长:42小时17分
- 粉丝8
- 金钱3110
- 威望80
- 积分4330