CEAC data is available and we are scraping!!!

Discussion in 'Lottery Visas - DV' started by Britsimon, Jan 1, 2018.

  1. saabe

    saabe Active Member

    @Britsimon
    I see you have some doubts about africa. Maybe density higher than usual?
     
  2. DanE1978

    DanE1978 Member

    I've been looking at Xarthisius' great graphs, as well as at the scrapped raw data, and noticed something a bit strange about Asia, as these seems to be a significant spike in the "ready" cases, especially compared to other areas, in which it is much less felt. Looking at the raw data, it seems like much of the ready cases are concentrated in embassies affected by the ban - Abu Dhabi, Asian numbers in Yerevan, Djibouti and Ankara, and the like. I wonder whether this may be the result of people from banned countries arriving at the embassy but being dissuaded from attending their appointments, or arriving at the embassy and being told not to pay the fees, somehow putting their case on hold?
     
    Sihem likes this.
  3. SusieQQQ

    SusieQQQ Well-Known Member

    Interesting observation. On the assumption that “ready” as usual means ready for interview, I wonder if it’s just that they haven’t bothered with the usual processing, background checks etc, and have decided that for travel ban countries they’ll only do those checks etc on AP after interview, where an applicant arrives who meets the criteria to circumvent the ban.
     
  4. Xarthisius

    Xarthisius Active Member

    I'm not sure that what you're seeing is real. Keep in mind that scale is different between regions: AF and EU are binned by 1000, AS is binned by 200. This makes all the "features" look wider on the latter.

    Most cases are concentrated in Abu Dhabi, Yerevan and Ankara, because over 35% of all people selected in AS region are from Iran and Yemen, which don't have their own embassies. UAE, Turkey and Armenia are the closest neighbor countries where those people can be interviewed. If you look at Kathmandu, it has almost the same number of ready cases as ANK, YRV and ABD combined...
     
  5. Xarthisius

    Xarthisius Active Member

    The only sad and outstanding thing about AS so far is the refusal level. Mid-January we reached number of people that were denied visa in AS, that's close to the final number for FY17. I'm afraid that it's only going to grow...
     
  6. SusieQQQ

    SusieQQQ Well-Known Member

    Travel ban the main reason?
     
  7. Xarthisius

    Xarthisius Active Member

    That I cannot answer with certainty, but I don't have any other suggestions...
     
  8. DanE1978

    DanE1978 Member

    Of course - the question is the ratio between the numbers. My laptop stopped being able to process the file due to the amount of date on it, but from what I managed to see, in addition to a relatively very high percentage of AP and refusals, the AS cases in these embassies also have a very high ratio of ready cases, compared to other embassies.

    You graph is also very telling - while EU and AF maintain a relatively stable ratio of ready/issued cases, being together around 40% of all cases, in AS the ratio begins to soar at some point, adding what appears to be 20% of the cases to the mix and reaching over 60% of the cases being registered as ready! From what I managed to notice the increase in ratio was mostly traceable to banned countries, but again, the file stopped responding before I could manipulate the numbers further.

    This would also explain what actually seems to be a relatively low rate of refusals. Given that around 40% of cases in Asia are from countries that are banned, and even assuming, for the sake of argument, that half of the applicants are somehow exempt from the ban, which seems to be a wild exaggeration, the refusal rate still doesn't seem to reflect that fact.

    All this is obviously just speculation, but people have been saying that when selectees from banned countries arrive at the embassy, they are told that they shouldn't take the interview since they will be paying the fees in vain. I wonder if this results in a refusal, or if the embassies refrain from registering it as such.

    BTW, I wonder if its possible to clean up the second set of graphs, so it only reflects cases from the actual region in which the embassy is. It obviously still wouldn't be precise, but it may be a little more accurate portrayal of the effects of the ban on Asia.
     
    Sihem likes this.
  9. DanE1978

    DanE1978 Member

    In Abu Dhabi, for instance, the ready cases outweigh the issued cases 4:1, while in Kathmandu, the ratio is 1:1.5 in favor of issued cases.
     
    SAM0130 likes this.
  10. SusieQQQ

    SusieQQQ Well-Known Member

    For that, it would be helpful if we had selectees that had happened to post & tell us what their CEAC status says after this has happened. I’m thinking that in a case where there is no actual interview resulting in an actual visa denial, it stays at ready. With all the court challenges and evolution of the ban over time, it’s also possible that the embassies are hedging in case those cases are actually able to be processed by fiscal year end if something changes. If it’s refused, it’s case over. If it’s just not interviewed yet, technically at least there is still a chance of rescheduling and issuing should the legal background change,
     
  11. Xarthisius

    Xarthisius Active Member

    You're forgetting about the fact that it's January. As per November'17 VB the limit for Iran is 2800. If you look at that range for e.g. ABD (number of cases):
    Issued: 59
    Refused: 82
    AP: 46
    Ready: 69

    There's a high chance that some of those 69 Ready cases (in range AS1-AS2800) belong to Iran and Yemen, but I'd hardly call it "hedging". All I can see is unprecedented number of refusals...

    It's true that there's a high number of 'Ready' cases above AS2800. Even if they're Iranians they couldn't have been processed yet...
     
  12. DanE1978

    DanE1978 Member

    I agree that this is just guesswork, but I've had a chance to go through some of the numbers, and unless I'm missing something, they are beyond crazy:

    In Yerevan, Asian cases are 45 issued, 123 AP 225 ready 80 refused and 5 refused;
    In Djibouti, Asian cases are 0 issued (!!) 25 AP 129 ready and 36 refused;
    In Ankara, Asian case are 67 issued, 56 AP, 157 ready and 44 refused;
    In Abu Dhabi, its 112 issued, 298 AP, 437 ready and 106 refused.

    In total, its 224 issued, 298 AP, 948 ready, and 226. The ratio between ready cases and issued cases is 1:4.23

    In contrast, the numbers for the rest of Asia, meaning embassies unaffected by the ban, are 1173 issued, 271 AP, 1067 Ready, and 111 refused. The ration between ready cases and issued cases is 1:0.9!!

    I'm no scientist, but these numbers seem VERY indicative of something. For whatever reason, the "banned" cases produce far far more "ready" cases. You guys know far better than me what exactly "ready" could mean, but this definitely looks like an anomaly.
     
  13. Xarthisius

    Xarthisius Active Member

    I posted numbers from Abu Dhabi that are relevant above, please look at them.
    To push my argument even further: in range AS1-2100 for ABD there are only 26 Ready cases (out of 69 for <AS2800). We still have half of Jan to go, to process those 43 Ready cases in 2100-2800 range (assuming they're from Iran)...

    They do NOT.

    It only means that there hasn't been an interview yet.
     
  14. DanE1978

    DanE1978 Member

    Honestly, I'm not looking for an argument, but I'm not getting why you're not willing to accept that the numbers are very much different in these embassies. Even the case of Djibouti alone says that there is something strange about their use of the ready category, with NO visas issued to the people from Yemen (which is obviously the case), and 129 ready cases - and this includes plenty of numbers that have been ready for months . It is so markedly different from all other cases that it can be accidental.

    I'm not comparing absolute numbers, since I don't see the sense in that. I'm comparing the ratio of ready and issued cases, and this ratio is more than four times higher when it comes to the embassies I noted. I'm not referring to the ranges, I'm referring to the actual numbers that appear in your raw files. Where is my math wrong here?

    The fact that we still have half of January to go is true for all other embassies as well. Why would it effect the ratios?

    The question is what does it mean that they were not interviewed, and whether, as SusieQQQ suggested, it could come to mean something different for the embassies in question, given their unique situation.
     
  15. Xarthisius

    Xarthisius Active Member

    I'm not looking for an argument either, but I don't like a situation when someone is presenting data and drawing conclusions that are simply impossible to draw.

    Good, let's focus on Djibouti as an example. First of all, there's no way to draw a conclusion from CEAC data about nationality of people assigned to a given embassy, because it's simply not there. It's a plausible assumption, I give you that, that majority of them may be from Yemen. Now, I don't know how you came up with 129 cases that are Ready (and all the other numbers you quote for that matter). There are exactly 69 cases and that's a fact that anyone can verify with the publicly available data. 26 of those 69 cases became ready on 24th December, when 2NL for February were sent. 20 of those 26 are actually above cutoff for January. These perfectly normal cases waiting for February interview. That leaves us with 43 mystery Ready cases. Now 20 of the remaining 43 are above December cutoff. These are cases that are being processed THIS MONTH. At this point we are left with 23 cases that are in Ready state for some reason. Does your ratios still look suspicious at this point?

    Maybe that's the problem. You operate under assumption that your ratios are somehow more meaningful. In reality they're just numbers taken out of context.

    Honestly, I don't even know how are you calculating those case numbers (69 vs 129??). I leave the burden of verifying our counts to the other readers.

    What you're trying to prove is that something extraordinary is happening in AS region. Saying that embassies are magically shielding people and that
    Where did you hear that? Certainly, there's nothing about that in CEAC data.

    One additional remark: I'm not a selectee from AS region. I have nothing at stake in being right or wrong here. I'm just very careful in drawing conclusions from non-trivial dataset.
     
  16. Xarthisius

    Xarthisius Active Member

    Oh! I think I know how you got that 129 number: you're summing the numbers in the "Ready" column, not the number of cases. As a result your ratios are really comparing the average size of family between different embassies.
     
  17. Britsimon

    Britsimon Super Moderator

    Some comments to add to the discussion.

    Ready cases could be any of the following scenarios (and a couple more rare ones):
    1. Someone scheduled in the future
    2. Someone who did not show up for their interview. They may reschedule, they may not.
    3. Someone who has been interviewed, and a decision has been made but the embassy has not yet updated their status.
    4. Someone who was on AP but Ready is shown temporarily before the status is updated.

    If a person is affected by the ban, they really should not be outright refused. They should be discouraged from having the interview, but if they want to be interviewed, they can do so and at that point (unless exempted by dual citizenship etc). they should be on AP. The travel ban is not permanent and could be removed by the government or by the courts. However, I do agree with Xarthisius that there may be some data that shows some cases (probably Iranian) are being moved to refused status. That is "good" for clarity and *perhaps* VB progress, but unfair to those cases, in my opinion. I would LOVE to hear from some of the Iranians who have knowledge of these things at the moment.
     
  18. DanE1978

    DanE1978 Member

    Right - I'm referring to visa numbers, not case numbers, simply because it was easier to aggregate them. So the only question is whether the derivative rate in those countries far exceeds the rest of Asian countries.

    I don't think there is much more to add about this point, which I agree might be incidental, although I think it is not. Even though I have an AS case, obviously that I don't have a stake in this debate either - whether this is right or wrong won't change the way things are actually done. Like most other people here, I'm trying to make sense of the publicly available information. I think I see something in the numbers that seems to explain why the ban doesn't have a much more significant effect on the numbers, like one would expect it to have. You think it is just incidental, and I can see why you say that.

    I can't remember where I read, either here or on Simon's blog, someone from the banned countries saying that they were told at the embassy not to take the interview because they would be paying the fee in vain. I do remember that it was a while ago, I think even for DV2017, when we still had the "bona fide" exemption.

    The one thing we know is that as of now, there are thousands of cases, mostly in Asia, that cannot get a visa. Its not a question whether this has an effect on the process, the real question is what kind of effect it has. It might be that these cases are not being processed, but I agree with Simon that this is not very likely. It might be that these cases are being processed, i.e., sent to the embassies, but interviews aren't taking place, either because they are not scheduled, or because people don't show up (or show up at the embassy and turned away). I imagine this scenario will be reflected in a growth in the "ready" cases, which I think is something we can see in the data. It can also be that people are being interview as usual, and either being refused or being put on AP (or somehow held on ready). Now there is a growth in the AP and refusal rates, but it doesn't seem significant enough to reflect the fact that a very large number of the cases processed thus far simply cannot get a visa. I think it makes more sense to assume (guess) that this is a result of the "extreme vetting" that regardless of the ban sends much more people, particularly from the countries in question, to AP, and is more prone to refuse them visas. In other words, what I think we're seeing is that the people that aren't exempt from the ban are left on ready status (out of their on volition or for other reasons), and the people that are exempt are being subjected to AP and refused in higher rates than usual. Again, this is a guess, but I think it fits the data better than alternative scenarios.
     
  19. Xarthisius

    Xarthisius Active Member

    To be honest, I'm a bit sad that I can't get through to you. It's my failure as an educator and as a person that strives to provide informational data analysis and visualization in both personal and professional life. I sincerely hope that I don't do equally crappy job in projects that I'm paid for. To make matter worse, the next data release in a few days will most likely have a huge bump in "Ready" cases, which will only fuel your confirmation bias.

    I don't have any further, reasonable arguments to change your mind. I was hoping that DJI example would be sufficient. I can only suggest that we revisit this conversation on 09/30/2018, when everything will be clear.
     
  20. DanE1978

    DanE1978 Member

    Okay, I give up. I'm moving to Canada instead. Au revoir
     
    Sihem likes this.

Share This Page