Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Codidact Meta!

Codidact Meta is the meta-discussion site for the Codidact community network and the Codidact software. Whether you have bug reports or feature requests, support questions or rule discussions that touch the whole network – this is the site for you.

Comments on What kinds of data import can QPixel support?

Parent

What kinds of data import can QPixel support?

+3
−0

How granular can data-import be for QPixel (I realize the answer might be different later for Codidact), and can it be done more than once? We know it's possible to import "everything", but which of the following would be possible, and which can be done incrementally?

  • All open questions on a tag or tags?
  • All open questions scoring > N and all of their answers > M?
  • All posts by a particular user or users?
  • Specific questions (with answers)?

Suppose a site wants to start small, importing selected tags, to try us out. Can more imports be run later to pick up other tags?

Suppose a site decides to import content from QPixel users (the "let me bring my stuff" model). When new users create accounts, can their posts be imported too? (It would be fine for these imports to be batched and delayed.)

Suppose a site has some individual questions that wouldn't otherwise be included, e.g. historically-locked questions (which are closed) -- can they specify a list of post IDs or URLs or something?

Imports are from the last data dump; can imports be "topped off" through the API, or would anything newer than the data dump need to be copied in manually?

I'm not making requests here; I'm asking what's generally possible, to inform discussions with communities. (One in particular, but others will presumably want to know this too.)

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.
Why should this post be closed?

0 comment threads

Post
+2
−0

New & Improved Imports v3.0 is out now on a Codidact site near you.

It takes data from the data dump where it can, fills in from the API where it can't, and lets you select exactly which posts to import by giving it a SEDE query specifying the post IDs to bring over. You can import anything that you can specify with a SEDE query - so selecting by users, tags (or combinations of tags), dates, scores - whatever - is totally possible.

History
Why does this post require attention from curators or moderators?
You might want to add some details to your flag.

1 comment thread

General comments (6 comments)
General comments
AA ‭ wrote over 4 years ago

Who can authorize an import? A mod?

ArtOfCode‭ wrote over 4 years ago

@AA General community consensus is the usual request

AA ‭ wrote over 4 years ago · edited over 4 years ago

@Art I'm specifically interested in a tool that allows someone (a mod?) to easily bring in duplicate questions on demand. Something to help in this sort of situation https://judaism.codidact.com/questions/276714 If a mod sees a new Q that has an unimported duplicate on SE, how easy would it be to import it on demand and let the thread develop from there with our users' further input?

ArtOfCode‭ wrote over 4 years ago

Not quite on demand, @AA - import is a manual process that requires a developer to run it, but we're happy for folks to say "hey, please import post id X" and we'll pull it in next time someone's available to do it... it's not immediate, but can be done.

AA ‭ wrote over 4 years ago

Can you give me a sense on turnaround time? Is within 24 hours reasonable or not? (I'm not pressuring and know you're doing a huge amount of volunteer work. Just trying to understand what we're working with.)

ArtOfCode‭ wrote over 4 years ago

@AA If you'd asked me a week ago I'd've said yes... I'm back at work now, so realistically probably 2-3 days, I'd guess? Most will probably be shorter, but I'm not going to make promises I can't deliver :)