Bug Error when importing from [OzzModz] Social Groups

[eDaddi]

ahh ok.

I'm on 2.3.4 in dev where the OzzModz add-on doesn't work on the front end. When you click a group you see nothing, no threads no nothing.

Unfortunately if you read through the OzzModz forum, many people went live like this and are screwed.

Is it possible to compile your import code into a single script that migrates all groups from OzzModz to yours? OR at least a way to select the nodes to migrate from ACP -> Forums?

 

[Fillip H.]

ahh ok.

I'm on 2.3.4 in dev where the OzzModz add-on doesn't work on the front end. When you click a group you see nothing, no threads no nothing.

Unfortunately if you read through the OzzModz forum, many people went live like this and are screwed.

Is it possible to compile your import code into a single script that migrates all groups from OzzModz to yours? OR at least a way to select the nodes to migrate from ACP -> Forums?

Oh, I thought you were talking about the thread import. The OzzModz importer is in the AdminCP, under Tools -> Import data.

 

[eDaddi]

Ahh ok, thats my bad.

I am getting this:



Maybe because my Data folder is on Cloudflare R2?

 

[Fillip H.]

Possibly. I don't have any way to test that.

 

[eDaddi]

Possibly. I don't have any way to test that.

If you're familiar with setting that up I can give you read only access to my buckets if that helps

 

[Fillip H.]

If you're familiar with setting that up I can give you read only access to my buckets if that helps

I'm very much not. You should move it back to local storage for the purpose of the import process.

 

[eDaddi]

I'm very much not. You should move it back to local storage for the purpose of the import process.

no worries. I assumed you mean the 'groups' folder, I put that local and still get that same error. I'm not sure what it's looking for.

 

[Fillip H.]

I've moved this to a separate thread while I look into this further

 

[eDaddi]

If there is anything I can help test with let me know.

 

[eDaddi]

View attachment 10695

I'm still getting this error with the latest release. Not sure if that one was going to fix this problem but I noticed a 'closed ticket' message in the other thread so thought I'd post.

 

[Fillip H.]

It was not supposed to change anything related to this. The other ticket was closed because it was implemented

I'll post back when I have more information about this specific issue

 

[Fillip H.]

Sorry for the delay in getting back to you. I've dug into the code a bit more and I've found that the local data directory needs to contain the following directories to work:

• data/avatars
• data/attachments
• data/groups (Snog's Groups data)

I would advise to do the following:
1. Close the forum temporarily
2. Copy those three directories to your server
3. Restore the original data config in src/config.php
4. Complete the import
5. Re-open the forum

 

[DragonByte Technologies]

Hello @eDaddi,

We hope your ticket regarding DragonByte Social Groups has been addressed to your satisfaction. This ticket has now been scheduled to be closed.

If your ticket has not been resolved, you can reply to this thread at any point in the next 7 days in order to reopen the ticket, afterwards this thread will be closed.

Please do not reply to this thread if your ticket has been resolved.

Thank you.


- DragonByte Technologies, Ltd.

 

[eDaddi]

Posting to prevent closing as I have not had time to test this, I can get back on it

Sorry for the delay in getting back to you. I've dug into the code a bit more and I've found that the local data directory needs to contain the following directories to work:

• data/avatars
• data/attachments
• data/groups (Snog's Groups data)

I would advise to do the following:
1. Close the forum temporarily
2. Copy those three directories to your server
3. Restore the original data config in src/config.php
4. Complete the import
5. Re-open the forum

Sorry for the delay, just getting time to play with this.

That makes total sense, but I'm approaching a TB of data in my buckets and don't have the space for it on my server. I'd bet there are many forums with similar setups.

I'm curious why the files actually need to be there, can you copy the DB info for that file and move it to your tables? I'm sure I'm missing something but just curious.

If they have to physically be on the server .... Can the importer DL what specific files needs to be there?

1. Loop through Song attachments
2. If exists locally, fine
3. If not copy to local, create folder if necessary
4. Run import
5. Clean up what was copied

Something to that effect .....Just brainstorming here

 

[Fillip H.]

The importer can't download from any buckets as far as I know, you'd run into the same problem if you were trying to import from one forum to another using the core XF importer.

You might need to skip importing attachments.

 

[eDaddi]

OK got to test it and after creating these empty folders:

• data/avatars
• data/attachments
• data/groups (Snog's Groups data)

I finally got to here:
View attachment 10733

Ran from CLI and I got:

php cmd.php xf:import
Starting import from [OzzModz] Social Groups (Beta)...
 - Step  1 of 17: Social groups             00:00:00 [13]
 - Step  2 of 17: Social group icons        00:00:00 [0]
 - Step  3 of 17: Social group banners      00:00:14 [10]
 - Step  4 of 17: Social group members      00:00:00 [546]
 - Step  5 of 17: Sections                  00:00:00 [0]
 - Step  6 of 17: Discussions               00:00:01 [217]
 - Step  7 of 17: Discussion replies        00:00:09 [4,741]
 - Step  8 of 17: Discussion reply edit history 00:00:00 [0]
 - Step  9 of 17: Discussion reply warnings 00:00:00 [0]
 - Step 10 of 17: Discussion attachments    00:00:04 [0]
 - Step 11 of 17: Discussion reactions      00:00:27 [4,876]
 - Step 12 of 17: Social group comments     00:00:00 [2]
 - Step 13 of 17: Social group comment replies 00:00:00 [26]
 - Step 14 of 17: Social group comment reactions 00:00:00 [10]

In AbstractStatement.php line 230:
     
MySQL query error [1062]: Duplicate entry '1' for key 'PRIMARY' 
                                                                  
xf:import [--processes PROCESSES] [--finalize]

Exception query:                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  INSERT  INTO `xf_mg_media_item` (`media_id`, `media_hash`, `title`, `description`, `media_date`, `last_edit_date`, `media_type`, `media_state`, `album_id`, `album_state`, `category_id`, `user_id`, `username`, `ip_id`, `view_count`, `watermarked`, `custom_fields`, `exif_data`, `warning_id`, `warning_message`, `position`, `imported`, `thumbnail_date`, `custom_thumbnail_date`, `poster_date`, `tags`, `last_comment_date`, `last_comment_id`, `last_comment_user_id`, `last_comment_username`, `rating_count`, `rating_sum`, `rating_avg`, `rating_weighted`, `reaction_score`, `reactions`, `reaction_users`, `featured`) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

 

[Fillip H.]

I'll need the full error from your server error log.

 

[eDaddi]

I can look when I get back, that query with the error is not showing in my previous reply.

I pasted this into notepad and forgot to send after the previous post:

When I unclick the 'Retain content IDs' option everything seems to work, both import AND the front end. I did have to find each attachment and pull them down from their bucket but all visible.

php cmd.php xf:import
Starting import from [OzzModz] Social Groups (Beta)...
 - Step  1 of 17: Social groups             00:00:00 [13]
 - Step  2 of 17: Social group icons        00:00:00 [0]
 - Step  3 of 17: Social group banners      00:00:14 [10]
 - Step  4 of 17: Social group members      00:00:01 [546]
 - Step  5 of 17: Sections                  00:00:00 [0]
 - Step  6 of 17: Discussions               00:00:00 [217]
 - Step  7 of 17: Discussion replies        00:00:08 [4,741]
 - Step  8 of 17: Discussion reply edit history 00:00:00 [0]
 - Step  9 of 17: Discussion reply warnings 00:00:00 [0]
 - Step 10 of 17: Discussion attachments    00:00:03 [0]
 - Step 11 of 17: Discussion reactions      00:00:27 [4,876]
 - Step 12 of 17: Social group comments     00:00:00 [2]
 - Step 13 of 17: Social group comment replies 00:00:00 [26]
 - Step 14 of 17: Social group comment reactions 00:00:00 [10]
 - Step 15 of 17: Group photos              00:01:10 [16]
 - Step 16 of 17: Group photo comments      00:00:00 [4]
 - Step 17 of 17: Group media               00:00:00 [0]
The data has been imported successfully.
Would you like to view any import notes? (y/n) y
+--------------------------------+--------+------------+
| Step                           | Total  | Time taken |
+--------------------------------+--------+------------+
| Social groups                  | 13     | 0 seconds  |
| Social group icons             | 0      | 0 seconds  |
| Social group banners           | 10     | 14 seconds |
| Social group members           | 546    | 1 seconds  |
| Sections                       | 0      | 0 seconds  |
| Discussions                    | 217    | 0 seconds  |
| Discussion replies             | 4,741  | 8 seconds  |
| Discussion reply edit history  | 0      | 0 seconds  |
| Discussion reply warnings      | 0      | 0 seconds  |
| Discussion attachments         | 0      | 3 seconds  |
| Discussion reactions           | 4,876  | 27 seconds |
| Social group comments          | 2      | 0 seconds  |
| Social group comment replies   | 26     | 0 seconds  |
| Social group comment reactions | 10     | 0 seconds  |
| Group photos                   | 16     | 70 seconds |
| Group photo comments           | 4      | 0 seconds  |
| Group media                    | 0      | 0 seconds  |
+--------------------------------+--------+------------+
| Total run time                 | 2.08 minutes        |
+--------------------------------+--------+------------+

A database table mapping old IDs to new IDs has been created: import_log_snogsocialgroups_1.
You may need this table for redirection scripts.

To finalize this import, you must continue via the control panel, or run the xf:import-finalize command.

php cmd.php xf:import-finalize
Finalizing import from [OzzModz] Social Groups (Beta)...
Rebuilding... Albums (1105)
Rebuilding... Media items (3682)
Rebuilding... Media items (7668)
Rebuilding... Media items (11491)
Rebuilding... Media items (15169)
Rebuilding... Media items (18625)
Rebuilding... Media items (21232)
Rebuilding... Media items (24619)
Rebuilding... Media items (28333)
Rebuilding... Media items (32401)
Rebuilding... Media items (36130)
Rebuilding... Media items (39750)
Rebuilding... Media items (43495)
Rebuilding... Media items (48301)
Rebuilding... Media items (51858)
Rebuilding... Media items (55711)
Rebuilding... Media items (59180)
Rebuilding... Media items (62938)
Rebuilding... Media items (66828)
Rebuilding... Media items (70711)
Rebuilding... Media items (74407)
Rebuilding... Media items (77650)
Rebuilding... Media items (82052)
Rebuilding... Media items (86188)
Rebuilding... Media items (90645)
Rebuilding... Media items (94532)
Rebuilding... Media items (98601)
Rebuilding... Media items (102650)
Rebuilding... Media items (106704)
Rebuilding... Media ratings (6775)
Rebuilding... Media ratings (15779)
Rebuilding... Media ratings (24786)
Rebuilding... Media ratings (34293)
Rebuilding... Media ratings (43444)
Rebuilding... Media ratings (53593)
Rebuilding... Media ratings (62981)
Rebuilding... Media ratings (72099)
Rebuilding... Media ratings (80533)
Rebuilding... Media ratings (89682)
Rebuilding... Media ratings (97816)
Rebuilding... Media ratings (106704)
Rebuilding... Album thumbnails (4)
Rebuilding... Album thumbnails (14)
Rebuilding... Album thumbnails (22)
Rebuilding... Album thumbnails (29)
Rebuilding... Album thumbnails (35)
Rebuilding... Album thumbnails (42)
Rebuilding... Album thumbnails (49)
Rebuilding... Album thumbnails (56)
Rebuilding... Album thumbnails (63)
Rebuilding... Album thumbnails (71)
Rebuilding... Album thumbnails (83)

Would you like to view any import notes and complete the import? (y/n) y
+--------------------------------+--------+------------+
| Step                           | Total  | Time taken |
+--------------------------------+--------+------------+
| Social groups                  | 13     | 0 seconds  |
| Social group icons             | 0      | 0 seconds  |
| Social group banners           | 10     | 14 seconds |
| Social group members           | 546    | 1 seconds  |
| Sections                       | 0      | 0 seconds  |
| Discussions                    | 217    | 0 seconds  |
| Discussion replies             | 4,741  | 8 seconds  |
| Discussion reply edit history  | 0      | 0 seconds  |
| Discussion reply warnings      | 0      | 0 seconds  |
| Discussion attachments         | 0      | 3 seconds  |
| Discussion reactions           | 4,876  | 27 seconds |
| Social group comments          | 2      | 0 seconds  |
| Social group comment replies   | 26     | 0 seconds  |
| Social group comment reactions | 10     | 0 seconds  |
| Group photos                   | 16     | 70 seconds |
| Group photo comments           | 4      | 0 seconds  |
| Group media                    | 0      | 0 seconds  |
+--------------------------------+--------+------------+
| Total run time                 | 2.08 minutes        |
+--------------------------------+--------+------------+

A database table mapping old IDs to new IDs has been created: import_log_snogsocialgroups_1.
You may need this table for redirection scripts.

The import has been finalized and completed and is ready for use.

A database table mapping old IDs to new IDs has been created: import_log_snogsocialgroups_1.
You may need this table for redirection scripts.

Does that mean I need to create a redirection script? Can those be inserted automagically? I'd hope to not have to do that when that query is fixed so nobody looses any SERP traction they may have gained.

 

[Fillip H.]

That message is automatically added by XF, but no, you won't be able to retain any URLs from the old social groups add-on.

 

[DragonByte Technologies]

Hello @eDaddi,

We hope your ticket regarding DragonByte Social Groups has been addressed to your satisfaction. This ticket has now been scheduled to be closed.

If your ticket has not been resolved, you can reply to this thread at any point in the next 7 days in order to reopen the ticket, afterwards this thread will be closed.

Please do not reply to this thread if your ticket has been resolved.

Thank you.


- DragonByte Technologies, Ltd.