[mod] Quick Hack. Importing without file uploading

3rd party code for phpList

Postby Hadcom » 1:31pm, Fri 17 Mar, 2006

Briliant , absolutely BRILIANT!!
Hadcom
phpLister
 
Posts: 6
Joined: 7:27am, Fri 17 Mar, 2006

Postby Lappert » 5:40pm, Thu 07 Dec, 2006

Question on versions.

I'm now running the current 2.10.3, but started with 2.10.2.

The 2.10.2 version of importcsv.php was 32,856 bytes, but the 2.10.3 version is 33,019 bytes.

The mod I downloaded is 38,729 bytes, but appears to have been written for a 2.8.x version of phpList.

Before I go and change everything (yes, I keep backups), I'm wondering if the current version of the mod works with 2.10.3 and does it implement changes that have occurred since the mod was written, ostensibly for the 2.8.x version.

Thanks
Lappert
PL Nut
 
Posts: 44
Joined: 3:32pm, Thu 14 Sep, 2006

Postby Lappert » 2:04pm, Fri 08 Dec, 2006

Well I sat down and spend a bit of time with this. Seems there are two versions of this, the second being more recent and written for 2.8.x. Some one said it would work with 2.10.2, but I compared the htaccess, import1.php and importcsv.php files to the current 2.10.3 versions.

While the .htaccess mod is simple enough, the other files are, IMHO, too different from the current 2.10.3 versions.

While I was able to get the textentry screen up, I'm very wary about implementing this mod into my system given that large parts of the file are different from 2.10.3. In other words, the mods were written for 2.8.x and I don't think it's wise to just plop the mods into a 2.10.3 system.

If the author(s) could update it to 2.10.3, I'm sure I and others would be appreciative.

I'm also unclear if you need both files. Seems the original hack was in import1.php, but the second hack uses importcsv.php.

A little clarification would help. Thanks.
Lappert
PL Nut
 
Posts: 44
Joined: 3:32pm, Thu 14 Sep, 2006

How to import a file

Postby alfredoyong » 6:30pm, Mon 22 Jan, 2007

Modifying code is too version dependent. We have to: first option, allow to use a memo field to import emails in the next version, or: second option, explain better how to import a file.

To import a a simple list, use excel to save it as "Text delimited by tabs" (don't forget to put the word "Email" as the first line in the sheet). Then import it to phplist using the option "import emails with different values for attributes"

Hope this helps.
alfredoyong
phpList newbie
 
Posts: 1
Joined: 6:24pm, Mon 22 Jan, 2007

Postby Lappert » 7:40pm, Mon 22 Jan, 2007

or: second option, explain better how to import a file.

To import a a simple list, use excel to save it as "Text delimited by tabs" (don't forget to put the word "Email" as the first line in the sheet). Then import it to phplist using the option "import emails with different values for attributes"


I think people know that (or they should). But that method is exactly the reason why I want to import on-the-fly from a list pasted within the program.

The excell method (although any text editor will do it) is a hassle unless you have a long list. If I want to import around ten emails, I don't want to deal with all the fuss. I just want to plop them in.
Lappert
PL Nut
 
Posts: 44
Joined: 3:32pm, Thu 14 Sep, 2006

Code request

Postby mmaywood » 10:35pm, Tue 22 May, 2007

Hi can someone please post this fix as something I can download and install.. my ISP will not allow me to have write access outside of FTP to the web server.. so I can not currently import contacts, the paste option seems to be the job.. using latest version 2.10.4 Thanks Mike (new to this)
mmaywood
phpList newbie
 
Posts: 1
Joined: 10:25pm, Tue 22 May, 2007

updated version for phplist - version 2.10.5

Postby manuel » 11:10pm, Tue 05 Feb, 2008

Thanks mariovaldez & jotazzu for the great work!

Maybe this is something that could be incorporated in the general releases so this solution doesn't break down in the future?

I was experiencing problems with uploading cvs files to import users (I believe it's because of php safe mode...)
If this is the correct, this modification could prove to be a very popular solution for a common problem...

I've only updated the "Deactivated original code" and some of the instructions to allow users of phplist 2.10.15 to easily implement this fix...

Modifications at public_html/lists/.htaccess (I didn't make any changes here...)
Code: Select all
# 'php_value' is wrong for magic_quotes_gpc because it's a logical value.
# Instead php_flag has to be used. This may be necessary only with older
# versions of PHP. PHP 5.x seems to be more tolerant or ignore it.
# (see PHP manual, section 9. Runtime Configuration, Running PHP as an Apache module)
# php_value magic_quotes_gpc on
php_flag magic_quotes_gpc on

# If you have changed the folder for temporary files in config.php you may
# have to adjust the path for the session information too. This may be a
# problem of older versions of PHP (lower 4.3?).
# (see PHP manual, section Session Functions)
php_value session.save_path "/PathToYourHttpRootFolder/tmp"

# 'register_globals on' should be used with versions of PHP lower than 4.3,
# because PHPlist uses the $_SESSION array as well as the functions
# session_register(), session_is_registered() and session_unregister().
# (see PHP manual, section Session Functions)
php_flag register_globals on

# It is not possible to switch the safe_mode in .htaccess files.
# (see PHP manual, section 9. Runtime Configuration, Running PHP as an Apache module)
# php_admin_flag safe_mode off


Please insert the following comment at line 2 of admin/commonlib/pages/importcsv.php
Code: Select all
### JT: Comment
#  This file has been modified to import csv-address-data not with browser
#  file upload from a loacal file but from an added <textarea> field in the
#  HTML form of the import2 page. Simply paste the csv-file in the textarea
#  field of your browser. Then continue as if you have set an upload file. I
#  have tested it successfully with a file size of 142 KB / 1800 addresses.
#  But I don't know the limitations in size of this upload method because the
#  data has to be stored temporarily in a session variable of the global
#  array $_SESSION. Normally the session array is stored as a file in the
#  configured tmp/ folder of the server. Therefore size limitations should be
#  rather high. But there may be other limitations.
#
#  This is a convenient way to keep that data persistent for more than one
#  page request without file uploads into the server file system. It is
#  necessary because there may be new additional colums and the import process
#  has to be interrupted. A page has to be shown where the user can assign the
#  new fields to existing ones. After this the import of data has to be repeated
#  and can be loaded from the session variable.
### /JT: Comment

paste the following code after the #if (ini_get("open_basedir")) block.
Code: Select all
##### JT: Modified code part1 for import via <textarea> begins
  # This should resolve a problem with older versions of PHP (lower than 4.3?).
session_start();
##### /JT: Modified code part1 for import via <textarea> ends

Then deactivate the first lines of function clearImport()
Code: Select all
### JT: Deactivated original code part1
#  if (is_file($_SESSION["import_file"])) {
#    unlink($_SESSION["import_file"]);
#  }
### /JT: Deactivated original code part1

You will find the next block of code to deactivate right underneath $_SESSION["test_import"] = $test_import;
Code: Select all
### JT: Deactivated original code part2
#  if(!$_FILES["import_file"]) {
#    Fatal_Error($GLOBALS['I18N']->get('File is either too large or does not exist.'));
#    return;
#  }
#  if(empty($_FILES["import_file"])) {
#    Fatal_Error($GLOBALS['I18N']->get('No file was specified. Maybe the file is too big? '));
#    return;
#  }
#  if (filesize($_FILES["import_file"]['tmp_name']) > 1000000) {
#    # if we allow more, we will certainly run out of memory
#    Fatal_Error($GLOBALS['I18N']->get('File too big, please split it up into smaller ones'));
#    return;
#  }
#  if( !preg_match("/^[0-9A-Za-z_\.\-\/\s \(\)]+$/", $_FILES["import_file"]["name"]) ) {
#    Fatal_Error($GLOBALS['I18N']->get('Use of wrong characters in filename: ').$_FILES["import_file"]["name"]);
#    return;
#  }
### /JT: Deactivated original code part2

##### JT: Modified code part2 for import via <textarea> begins
  if(!($_POST["import_elist"])) {
    Fatal_Error("No list was specified.");
    return;
  }
##### /JT: Modified code part2 for import via <textarea> ends


Some lines further down you will find the third part of the code that needs to be replaced
Code: Select all
### JT: Deactivated original code part3
#  if ($_FILES["import_file"] && $_FILES["import_file"]['size'] > 10) {
#    $newfile = $GLOBALS['tmpdir'].'/'. $_FILES['import_file']['name'].time();
#    move_uploaded_file($_FILES['import_file']['tmp_name'], $newfile);
#    $_SESSION["import_file"] = $newfile;
#    if( !($fp = fopen ($newfile, "r"))) {
#      Fatal_Error(sprintf($GLOBALS['I18N']->get('Cannot read %s. file is not readable !'),$newfile));
#      return;
#    }
#    fclose($fp);
#  } elseif ($_FILES["import_file"]) {
#    Fatal_Error($GLOBALS['I18N']->get('Something went wrong while uploading the file. Empty file received. Maybe the file is too big, or you have no permissions to read it.'));
#    return;
#  }
### /JT: Deactivated original code part3

##### JT: Modified code part3 for import via <textarea> begins
  # Set the import file name with the import data itself from the textarea field.
 
  if (!$_SESSION["import_file"]) {
    $_SESSION["import_file"] = $_POST["import_elist"];
  }
##### /JT: Modified code part3 for import via <textarea> ends

you will find the next block of code to deactivate a couple of lines below if ($_SESSION["import_file"])
Right underneath the flush...
Code: Select all
### JT: Deactivated original code part4
#  $fp =  fopen ($_SESSION["import_file"], "r");
#  $email_list = fread($fp, filesize ($_SESSION["import_file"]));
#  fclose($fp);
#  flush();
### /JT: Deactivated original code part4

##### JT: Modified code part4 for <textarea> import begins
  $email_list = $_SESSION["import_file"];
##### JT: Modified code part4 for <textarea> import ends

Part5 solves a problem in definition of the numbers of imported colums
Code: Select all
### JT: Deactivated original code part5
#    if (sizeof($values) != (sizeof($_SESSION["import_attribute"]) + sizeof($_SESSION["system_attributes"]))
#      && $test_import && $_POST["show_warnings"])
#      Warn("Record has more values than header indicated (".
#        sizeof($values). "!=".
#        (sizeof($_SESSION["import_attribute"]) + sizeof($_SESSION["system_attributes"]))
#      ."), this may cause trouble: $index");
### /JT: Deactivated original code part5

##### JT: Modified code part5 for import via upload file begins
    # The original comparison ignores that sizeof($values) also contains the field 'Email' which
    # is not contained in 'import_attribute' and 'system_attributes'.
    if ((sizeof($values)-1) != (sizeof($_SESSION["import_attribute"]) + sizeof($_SESSION["system_attributes"]))
       && $test_import && $_POST["show_warnings"])
      Warn("Record has more values than header indicated (".
         (sizeof($values)-1). "!=".
        (sizeof($_SESSION["import_attribute"]) + sizeof($_SESSION["system_attributes"]))
      ."), this may cause trouble: $index");
##### /JT: Modified code part5 for import via upload file ends

At last part 6 is HTML code with fragments of PHP. It generates the textarea field in the HTML form for import.
Code: Select all
<!--
### JT: Deactivated original code part6
#<tr><td><php>get('File containing emails')?>:<br>
#</td><td><input>
#<br><php>get('uploadlimits'),ini_get("post_max_size"),ini_get("upload_max_filesize"));?>
#</td></tr>
### /JT: Deactivated original code part6
-->
<!-- ##### JT: Modified code part6 for <textarea> import begins -->
<tr>
  <td>List of emails:<br></td>
  <td>
    The following limits are set by your server:<br>
    Maximum size of a total data sent to server: <b><ini_get></b><br>
    Maximum size of each individual file: <b><ini_get></b><br>
    PHPlist will not process files larger that 1Mb
  </td>
</tr>
<tr><td><textarea></textarea></td></tr>
<!-- ##### JT: Modified code part6 for <textarea> import ends -->
manuel
phpList newbie
 
Posts: 3
Joined: 9:07pm, Tue 05 Feb, 2008

Postby coeurmechante » 11:06pm, Sun 23 Mar, 2008

I'm trying to modify importcsv.php to allow a user to enter a path to the file to import on a server via text form. I'm trying to do so based on your excellent work getting the textarea form in place, but am not sure if that should be my springboard point.

I'm close, but not there yet (still wading through the import code, which is slightly beyond my ken, though I'm catching up). Wondering if anybody else has managed to accomplish anything like this?

I have cron jobs set up to make daily updates to a master import list which is tossed on the server, and it'd be a big client time-savings as I could automate the whole thing. If I figure this out, would it be useful to anyone else?
coeurmechante
phpLister
 
Posts: 5
Joined: 6:35pm, Wed 12 Mar, 2008

Postby Thales Piza » 3:07pm, Tue 24 Jun, 2008

bevan wrote:I just answered your other questions about this here:
ww[dot]phplist[dot]com/forums/viewtopic.php?p=15117#15117
ww[dot]phplist[dot]com/forums/viewtopic.php?p=15116#15116


Both links are dead now.. How can I do this importing without using files??
Im running my server on an external host... Didnt saw any admin folder...
What should I do here??

This is a problem quite often, would be a very nice idea to change it in the next release... to allow adding 1+ members at the same time...
Any ideas??
thx
Thales Piza
phpLister
 
Posts: 7
Joined: 6:33pm, Mon 23 Jun, 2008

Postby jvarrone » 11:33pm, Mon 28 Jul, 2008

coeurmechante wrote:I'm trying to modify importcsv.php to allow a user to enter a path to the file to import on a server via text form. I'm trying to do so based on your excellent work getting the textarea form in place, but am not sure if that should be my springboard point.

I'm close, but not there yet (still wading through the import code, which is slightly beyond my ken, though I'm catching up). Wondering if anybody else has managed to accomplish anything like this?

I have cron jobs set up to make daily updates to a master import list which is tossed on the server, and it'd be a big client time-savings as I could automate the whole thing. If I figure this out, would it be useful to anyone else?


I'd REALLY like to be able to do this - import a list via cron that had previously been dropped on the server. Basically run the script every hour, if the file is there, import it (using appropriate command line parameters to answer the import user questions).
jvarrone
phpLister
 
Posts: 10
Joined: 2:32am, Sun 21 Aug, 2005

Postby undercover » 5:32pm, Fri 15 Aug, 2008

coeurmechante wrote:I'm trying to modify importcsv.php to allow a user to enter a path to the file to import on a server via text form. I'm trying to do so based on your excellent work getting the textarea form in place, but am not sure if that should be my springboard point.

I'm close, but not there yet (still wading through the import code, which is slightly beyond my ken, though I'm catching up). Wondering if anybody else has managed to accomplish anything like this?

I have cron jobs set up to make daily updates to a master import list which is tossed on the server, and it'd be a big client time-savings as I could automate the whole thing. If I figure this out, would it be useful to anyone else?


YES!!! I'm trying to do the exact same thing. Basically, I think everything can be done in importcsv.php. Change the input type on the form for the file from file to text. Get rid of all the code that moves an uploaded file from the temp directory. Then, in the code that opens the file, reference the submitted path instead of the location where the uploaded file was moved from the temp dir. Everything else should work fine from there. I've just started messing with this, but I'll post my code if/when I get it working.
undercover
phpLister
 
Posts: 7
Joined: 9:07pm, Thu 19 Jun, 2008
Location: Seattle, WA

Postby undercover » 10:53pm, Mon 18 Aug, 2008

I changed the importcsv2.php file to import the CSV file locally rather than upload it. Just replace your importcsv2.php file with the attached one, and you're good to go. In your admin interface, just list the full location of your import file on your server - eg. '/home/userid/www/import/import_file.csv' - in the space formerly occupied by the upload textbox.

Note: this didn't really help me to a more efficient import process. Even without the 1mb upload limit, the csv import taxes the system such that I can't do more than 10k records without exceeding CPU limits on my shared hosting plan.

Edit - guess I can't attach stuff yet. PM me if you'd like a copy of the file.
undercover
phpLister
 
Posts: 7
Joined: 9:07pm, Thu 19 Jun, 2008
Location: Seattle, WA

Postby mredant » 11:06am, Tue 07 Apr, 2009

cant find this part in the source code;

you will find the next block of code to deactivate a couple of lines below if ($_SESSION["import_file"])
Right underneath the flush... <<-- which-one?

Code:
### JT: Deactivated original code part4
# $fp = fopen ($_SESSION["import_file"], "r");
# $email_list = fread($fp, filesize ($_SESSION["import_file"]));
# fclose($fp);
# flush();
### /JT: Deactivated original code part4

##### JT: Modified code part4 for <textarea> import begins
$email_list = $_SESSION["import_file"];
##### JT: Modified code part4 for <textarea> import ends
mredant
phpList newbie
 
Posts: 3
Joined: 10:21am, Fri 30 Jan, 2009

Postby manuel » 4:54pm, Tue 07 Apr, 2009

It's under the following flush:

Code: Select all
  print "<p>".$GLOBALS['I18N']->get('Reading emails from file ... ');
  flush();


Greetings,
Manuel
manuel
phpList newbie
 
Posts: 3
Joined: 9:07pm, Tue 05 Feb, 2008

Postby manuel » 5:18pm, Tue 07 Apr, 2009

I'm experiencing something strange concerning the posting of code in this forum.
In the post above I wanted to include a large piece of the code but noticed part of it wasn't posted correctly...

I have retried several times but failed
Apparently the forum parser is correcting some parts of the code!!!
This causes valid posts to contain invalid code!!!

Please try this for yourself and post the complete IF statement including the following code AND the print and flush statements directly below it: (there's only one in importcvs.php)


Code: Select all
for ($i=0;$i<10000; $i++) {


This is the result:
Code: Select all
if ($_SESSION["import_file"]) {
  # output some stuff to make sure it's not buffered in the browser
  for ($i=0;$i<10000>get('Reading emails from file ... ');
  flush();



Can you spot the problem???

Greetings,
Manuel
manuel
phpList newbie
 
Posts: 3
Joined: 9:07pm, Tue 05 Feb, 2008

PreviousNext

Return to Contributions: Plug-ins, Add-ons, Mods

Who is online

Users browsing this forum: No registered users and 1 guest

cron